In Part 2 we drew a triangle using a Vertex Buffer, and some basic shaders. While on the surface this can seem overtly complicated, it actually becomes the basis of a powerful OpenGL achitecture to enable you to leverage the GPU is a variety of very interesting ways without having to rely on the CPU.
This also comes from the "OpenGL's Moving Triangle" section of the Learning Modern 3D Graphics Programming online book.
With this code, we are going to take our original triangle, and we will make it move around a bit, and also change colour at the same time.
The clever thing about this is, we won't be changing the vertex data stored in the buffer, but will instead be manipulating it with Fragment and Vertex shaders. This almost feels like Uber-CSS over the top of HTML.
This is pretty powerful stuff, as we can let the GPU do a lot of the processing by using shader programs, and passing them attributes to control the overall affect that we want.
You can see we now have a uniform vec2 offset;. This defines a value that is going to get passed in from outside the Shader. It has the keyword uniform as the value stays the same on the same rendering frame.
The offset attribute will be expecting a vector with an x and y coordinate to be passed through (vec2) for the uniform value.
We convert the offset vec2 to a vec4, as you can't add a vec2 to a vec4. Then we can add these two together (GLSL will do vector arithmatic for you out of the box) to get our final gl_Position vector.
This means we can change the position of our triangle with relative ease, just by changing the offset of each of the vertices.
To do this from our JRuby code, is quite straight forward. The first thing we need to do is find out the location / position of the offset attribute. This is done through:
compute_position_offsets calculates x and y offsets based on the current time. (Check out the code for more details, it's just some fun trig). To set the uniform value, we then do:
You can see we can define constants with the const keyword. This sets up two colours to mix between, in this case, white and green.
We have two uniform attributes to in this time, fragLoopDuration the loop duration of the colour change, and time, how many seconds have passed since the application began.
mix is a GLSL function that blends two colours together based on the third float that is passed through, and gives us the slow fade between the two as the time value changes.
Setting this up is almost exactly the same as before. We will set the fragLoopDuration in our init_program method, as it never changes across the execution of our code.
For the time uniform attribute, we have some code that tracks how much time passes between frames and we just add it all up, and then set the uniform attribute with gl_uniform like usual:
In Part 1 we created a window, nothing too fancy. Now we get to actually display a triangle.
Just to follow along as well, I'm moving through the Learning Modern 3D Graphics Programming online book to learn OpenGL (again), so the OpenGL examples I will be displaying will be ports from the code that that is providing. I would suggest for the complete theory behind this code, read the linked section before going through the JRuby code. I expect my explanations to only be commentary on the information already provided in that series and discussion on some of the finer points on JRuby and Java library integration as well.
If you have any questions, please feel free to ask, but be aware, I'm very new to OpenGL, so writing this series is very much part of my learning experience. However, I will attempt to answer the best way I can. On the other hand, if you find anything wrong with what I'm written, please point it out so it can be corrected.
When I first did OpenGL back in University, we used the glBegin() and glEnd() paradigm. This was definitely far easier that the more modern APIs, as it was very clear and easy to draw a simple polygon on the screen (example). However, it did mean more computation was occurring on the CPU and a larger use of the system RAM than the newer APIs. The newer APIs, while (far?) more complicated, shift much of the work to the GPU and also provide a far more flexible implementation. I liken it to working with HTML and tables back in early HTML days. Sure it worked, but CSS and semantic markup gives a clear separation and creates far more flexible implementation options (at least in theory ;) ).
So we have some basic vertex information to display a right angle triangle:
Each line of this array define the x, y and z coordinates of our triangle. You will notice there is a fourth coordinate (1.0) on each line. This defines the clip space. For now we'll just say this just means that the vertexes you see in the window has to have values between -1 and 1 on the x, y and z axis. More than that will render outside of the window.
As discussed previously, in old school OpenGL you would just look through this list of vertexes and say draw a triangle here, however, this is no longer the case!
I feel like modern OpenGL is almost like a database - you put some data into it, and have an id to reference that data that was placed in. Then you can work on that data that is stored on the GPU through some other techniques (that we will look at in a minute) via that id. This seems to be a concept that is used across the board.
The code that inserts our vertex data into the GPU can be seen in the method init_vertex_buffers
So first thing we do, is we generate an id for the vertex buffer, which is where we will store our vertex data. (gl_gen_buffers). Then we tell OpenGL, hey, this is the buffer we want to work with for the moment through gl_bind_buffer, passing in the specific @buffer_id we generated before. We also tell OpenGL that the buffer we are working with an GL_ARRAY_BUFFER, so it knows what data to expect.
In case you aren't aware, JRuby will convert Java static constants to Ruby constants, so we can access these static fields very easily.
To pass the vertex data into the new vertex array buffer, LWJGL has us use it's BufferUtils class to create a NIO buffer, and push the data into it, like so:
float_buffer = BufferUtils.create_float_buffer(@vertex_positions.size)
float_buffer.put(@vertex_positions.to_java(:float))
#MUST FLIP THE BUFFER! THIS PUTS IT BACK TO THE BEGINNING!
float_buffer.flip
A couple of interesting notes:
You will noticed the .to_java(:float). That is the JRuby code for converting a Ruby array to a Java array. Passing in :float tells it to make it an array of primitive floats.
The .flip at the end. This is very important (and took me a day to work out, as I'm not familiar with NIO buffers). Here is a great article that explains it more detail, but essentially the buffer tracks where it is at, and flip sends it back to the beginning. Without this, no data goes to our Vertex Array, and nothing happens!
After we are done, we tell OpenGL not to be bound to any array buffers (0 works like NULL in OpenGL land). This could be considered optional, but ensures that weird things don't occur.
Now all we have to do is actually write the code that makes the data that displays the triangle!
Telling OpenGL how to Render the Vertexes
So now we have the vertex data stored on the GPU, we have to tell it how to render it, and to do that, we have to build a program of a couple of different types of shaders. Think of Shaders kind of like the CSS of HTML. They simply work on the existing data in the GPU and tell it how it to render (although that's a bit of an over simplification).
First thing, we'll write a simple vertex shader in GLSL, the language for writing shaders. This specifies to the GPU where the vertexes actually are from the data you entered earlier. We'll just say that it's correct and basically pass it through.
To make life easier for myself, I wrote a quick little create_shader function, that loads the shader from a file loads it into memory and compiles it. I just want to make note of one line:
Without this, you have no idea why your shader fails (if it does). Mine was failing, and I didn't realise it until I looked deeper. (Version 330 of GLSL wasn't supported on my Ultrabook on Linux with Mesa. I had to switch to my main laptop with the Nvidia graphics card)
Make sure to look at the output logs!
Creating the program to define how our data is output on the screen, is quite similar to what we did before. We generate an id and then link the shaders to the program like so:
This tells OpenGL how the structure of the vertex buffer data matches is structured. Here we are saying that we have an array of floats, and every 3 elements is a x, y and then z member, with the fourth defining the clip space, as we saw earlier. (0 is the start, 4 is the size).
GL11.gl_draw_arrays(GL11::GL_TRIANGLES, 0, 3)
DRAW THE TRIANGLE! 0 is the start of the array of vertices you want to draw, and 3 is the number of indexes you want to process - in this case 3 to make a triangle.
Warning: This is going to be far less technical, and more about the journey of learning that I'm enjoying.
As per my previous blog post, and from several tweets I've posted, I've been slowly plodding away on doing some game development in my spare time and in my holiday break. I originally started doing this because I've always loved games, and was deeply inspired by the indie game development scene and some of the incredible ideas that were coming out of that.
That also being said, I was really keen to try a new programming challenge. Something totally outside of my usual purview, and doing game programming seemed like a perfect fit for that, as it sat so far outside my usual environment of building web based business applications. What I didn't realise was how much it was going to push me in both directions that we new and also topics that I've learnt (far?) in the past and unfortunately forgotten much about.
One aspect that has totally blown me away was the architectural idea of "Entity Systems" (ES) for game development. (I'm not going to explain it here, there are way too many good articles on this, but if you want to read more click the link above). For what is such a relatively new idea in software development (< 10 years old if what I believe is correct), it fits so incredibly perfectly into how to write some insanely flexible game architectures, I have been rolling it around inside my head for other ideas of where it can be applied.
Since the last blog post, I refactored my entire code base into an ES architecture, and that was a fair bit of a mental shift away from a traditional OO model, but the more I use it, the more it just makes an incredible amount of sense. From the screen shot below you can also see I started adding some graphics to the page too, to make a little nicer to look at.
Some of the basics (and beginnings) for this came from the awesome SpriteLib project, but I also decided to start doing some drawing and try my hand at doing some of my own pixel art. Most people probably don't know this, but I used to draw a lot. I even spent a year in design school... but to be honest, I was nothing special at it and ended finding a real passion for programming. But doing this game programming stuff has got me sketching out little characters and whatnot again, so it's been a real pleasure to rediscover this passion for drawing that I haven't touched in probably over a decade.
My next step in the game is to do some basic physics, nothing too special, just a player and some jumping to move around the world. Suddenly I'm back in high school doing kinematics and dynamics! I've still got all my old maths and physics notes from high school, so I'm pouring over that and also reading some great websites as well, and the concepts and equations are slowly coming back to me.
While doing that though, I started to look at the development cycle of Slick2d and the fact that it seemed like many people (including the original developers themselves) where moving over to libGDX. Comparing Slick2D and libGDX, you can clearly see that Slick2d has a far lower barrier to entry that libGDX. From my review of libGDX, having some knowledge of OpenGL and how 3D graphics work (even on a 2D project) is going to be a huge boon. So now I'm wondering - do I switch to libGDX? It looks more active, performant and gives far more options for distribution (desktop, web, android).
Again, I'm back to looking at old code from University - specifically my 3D programming class, and trying to make heads and tails of the OpenGL code I had previously written (in C++ no less). It vaguely makes sense, but there are concepts there I have either partially or totally forgotten (what is a Quad again? And different projection modes? no idea). Therefore, I'm going to keep playing with OpenGL some more, just to get that basic foundation under my feet, starting with the very low level lwjgl project with JRuby, and just keep going forward one step at a time, and then probably refactor my code again into libGDX once I'm done. While I don't think I have to really know OpenGL to use libGDX, I always feel more confident having the base understanding under my feet. That and I think it's a direct path to re-learning concept as vectors and 3d math with matrices that really is core to any sort of advanced game programming (yeah, forgot most of that too).
So overall, it feels like I am going around in circles a bit - but not in a bad way. I'm finding old things that I used to love doing (drawing and math) and rediscovering the joy I had in doing them, and although it's frustrating knowing that you have forgotten the knowledge you wish you still knew, it's a delight to reopen these memories and play with them once again.
For something different, I decided I would try my hand at doing some game development programming. I was incredibly inspired by many of the indiegamesI watchbeing actively developed, and I figured it would be a interesting way to learn some brand new things totally outside of my usual purview.
I wanted to do some more work with JRuby, as I had a good hack with it at work, but haven't touched it in a while, so I hunted down which framework for doing a game. I've decided I wanted to do a desktop based 2D something (no idea what still), so settled on Slick2D as my framework of choice.
I've starting leaning towards doing something open world-ish, which meant I needed to generate out a large chunk of landscape without having to hand build a map, and yet also make sure it was the same every time a user came back to a particular world. That lead me to start investigating Perlin Noise, and I settled on the perlin_noise ruby library for generating that, and so far it's been very good.
It's been interesting too, having JRuby as the platform. It's a tough choice deciding whether to use a Java library or a Ruby one. With Slick2D it was easy - it was the best there was, and it hooks straight into OpenGL. With something like a Perlin Noise library, ease of use was the primary concern, and managing gems with Bundler and having a direct line to a nice Ruby syntax based API was the deal clincher. Although if it ever becomes a performance concern, I may look to go back to straight Java.
So here is a video of what I have so far. It's still very raw, but basically it's a circular big world, that if you go one way for long enough, you come back right where you started.
I'm extending Slick2D's Shape class to create my world object (basically a circle that I modify with my generated perlin noise). I've already set it up so the world has a "coordinate", and that controls what the world looks like. In theory, if this ever becomes a real game, you could give someone else your world coordinates and they would see the same world you do.
Before anyone asks, I've not decided what this is going to be, or if I'll open source it, or what (it's a bit of a sandpit at the moment), but I'll keep writing a diary here of what I've been doing with it, with appropriate links as long as it keeps staying interesting.
Got a few ideas on where to go next (zoom in and out, dynamic texturing of the landscape, etc), but haven't committed yet. This stuff is pretty interesting, and very different from the sort of stuff I code on a day to day basis.
Let me know if you find this interesting, and I'll keep writing about it!
Having worked with both Groovy, (J)Ruby and, Java and (to a much smaller degree) Clojure, I can pretty easily claim that ColdFusion's threading and distributed processing capabilities.. leave a little to be desired. (Okay, actually a whole lot).
There are solutions though - using a job queuing system like Amazon SQS, IronMQ, or if you are looking for a something more threaded, dig into Marc Esher's awesome cfconcurrent project.
But sometimes you just want to take an array of values and run them through some sort of processing in a threaded manner, without having to jump onto any other projects or infrastructures, and without having to write that nasty <cfthread> create and join pattern for the millionth time.
So I wrote a quick CFC I've called 'FutureThreadedWorker', which does exactly that, without half the code you would otherwise need. (Note: this was written for ColdFusion 9. It would be a bit nicer if I could use closures)
<cfscript> //create, passing in an array for the array of structures (which are the arguments passed to the workerMethod) //then pass it the worker to call the worker method on //tell it how many thread it's should use //finally, tell it the name of the method to invoke. var future = new services.util.FutureThreadedWorker(queue, this, 5, "doWork");
//start the running future.run();
//if I want the results, wait for them. var results = future.get();
//if i want to check for errors var errors = resuls.getErrors(); </cfscript>
Bam. That's it. The FutureThreadedWorker now loops around the passed in array, and passed in the arguments to the method specified on the worker without you having to worry about creating threads or joining them back up again. I think you'll find that a lot easier than using <cfthread>.
The new function, swithThreadContextClassLoader() is useful as it is often required when dealing with libraries, such as dom4j, that use the ClassLoader to define singletons or search for implementing classes, and don't allow you to overwrite what ClassLoader they use. Check out the wiki for more details on why this is neccessary, and how easy it is to now do, thanks to our new function!
Also, JavaLoader has been move to GitHub, along with the all the documentation. This should make collaboration much easier moving forward!
Otherwise, make sure to sign up to the mailing list, and enjoy loading your Java.
In ColdFusion 9, several Web Services come pre-built with the product, allowing greater interoperability between different technologies and ColdFusion.
The other day saw the release of my Dzone RefCard outlining how you could take advantage of these Web Services from Java, and use ColdFusion's
If you are doing Java development, and are looking to integrate some of ColdFusion's capabilities into your application, you can download the RefCard from here . It will step you through setting up the Services, generating the required Java code to run the Web Services, and also provides examples of usage to help you get started.
ColdSpring
2.0, codenamed 'Narwhal', is a project that has been going on a little
'behind the scenes', except for the occasional tweet from either myself, or Chris Scott.
In
this presentation, we'll look at some of the motivations behind the
complete rewrite of ColdSpring for the 2.0 version, and some of the new
features that will be available that should make dependency injection
easier, and way more flexible than ever before.
I've had in my
to-do list a reminder to write a long blog post on what is going on
with Narwhal, which I should write at some point soon, so you have a
good idea what to look out for. That being said, Narwhal is taking
shape nicely, and I think is going to be a very powerful addition to
the ColdFusion framework landscape.
Advanced Java & ColdFusion Integration with JavaLoader 1.0
This
talk will focus on the new features of JavaLoader 1.0, and how you can
use them to integrate Java with your ColdFusion platform in some new
and exciting ways. If you are interested in JavaLoader 1.0, check out
this previous blog post, it gives you a good run down on the new features it brings to the table.
We
will also be investigating some of more common 'gotchas' with Java and
ColdFusion integration, especially around ClassLoader issues, which
should be useful for all involved.
Bob Silverberg and I are also going to be a teaching a 2 day training course, just before cf.Objective():
This
will be as very hands on session, where Bob and I will be going through
building an application using ColdFuson 9's Object Relational Mapper
from beginning to end. This will include many best practices,
discussions as well as theory about how the underlying Hibernate engine
works with ColdFusion. No ORM/Hibernate experience necessary.
More detail on the course can be found on our website, and also on the preconference page, where you can also check out all the other fantastic training content.
Don't
forget that the early bird pricing for both conference tickets, and
training finishes on the 29th of January, which is coming up soon!
cf.Objective() is looking to be a great conference. Hope to see you there!
Credit and thanks to Fred Roeber for his contributions to the NetworkClassLoader in JavaLoader.
If you are interested in hearing more about ColdFusion and Java integration please come to cf.Objective(), where I will be presenting on Advanced Java & ColdFusion with JavaLoader 1.0.
In the mean time, download JavaLoader, and please report any bugs that you find.
In the interview we talk about a variety of topics, mainly centred about how and why to integrate ColdFusion and Java together, but also touching on things like Object Relational Mappers, including ColdFusion 9's integration with Hibernate.
Hope you like it, and please give it a nice 'up' on Dzone.
The was a logic bug in the way that dynamic compilation was occuring across multiple source directories that has now been resolved, and should also ensure that compilation is much faster than it was in the Alpha.
The other enhancement was to allow for relative paths in the Spring integration. To explain, in the Alpha, you would often end up having to input your beans similar to this: <coldfusion:cfc id="message" script-source="file://${root.path}/model/Message.cfc" script-interfaces="com.IMessage" />
Which can be a bit of a pain, as injecting a dynamic value for ${root.path} is not as easy in Spring as it is in ColdSpring.
Now, by default, 'script-source' attributes are relative to the calling page, so you can do your bean definitions like this: <coldfusion:cfc id="message" script-source="file:///model/Message.cfc" script-interfaces="com.IMessage" />
and the path will be worked out for /model/Message.cfc relative to your application path.
Oh, and don't forget - if you are into integrating Java and ColdFusion development, make sure you sign up for the Google Group !
I had the wonderful opportunity to present and attend MAX for the first time ever this year, and I can honestly say, it was a fantastic time all around.
MAX is completely unlike any other conference I've ever attended, both in terms of its sheer size, and also in terms of the wide gamut of Adobe community members that are in attendance. I managed to meet a variety of people that I would never have talk to in my regular travels both in term of physical locations and also technological disciplines.
I also had the pleasure of presenting the last session on the last day, so I was very happy when I had a fairly reasonable turn out for my ColdFusion for Java Developers presentation. The presentation took the perspective that the attendees were/are already doing Java development, but want to port those tools and codebases that they are already using to a ColdFusion environment, so they could take advantage of ColdFusion services and language. From there we look at the variety of techniques that you can use to enable ColdFusion->Java communication, as well as Java->ColdFusion. This includes a aan overview of a lot of the JavaLoader 1.0 feature set in the process.
If you are one of these people, and/or you are doing development in both ColdFusion and Java, you can watch the recording of my presentation on Adobe TV . You may want to skip the first 5 minutes, as for some reason the recording has me doing all my set-up before I start speaking.
At a very ANZ friendly time of Wednesday, 10am Melbourne time, I'm going to be joining super-smart guys Brian Kotek and Barney Boisvert on CFPanel to talk about Java Integration with ColdFusion in the CFPanel Connect Room.
This
is obviously a topic that is quite near and dear to my heart, but it is
also one that I think often doesn't get its 'surface scratched' (so to
speak). In my experience I often see the information presented is more
on the 'how to get set up' side, and less on the deeper details on what
can actually be done.
I'm really looking forward to getting a chance to share experience with the other CFPanel
members, and really get deeper into some interesting Java and
ColdFusion integration, in regards to how people are using it, what
benefits they are reaping, and also what problems they have encountered.
For more details, and and all the other time zones, check out the CFPanel post.
JavaLoader
1.0 is a project I've been talking about off and on for the past 6
months or so, and I'm very happy to announce that a release version is
now complete.
There are several key new features to JavaLoader 1.0,
Dynamic Compilation
Now
if you are working with JavaLoader, you don't have to bundle your Java
code into a jar to load. You can simply specify to JavaLoader which
directories are your source directories, and JavaLoader will compile
and load them on the fly for you! JavaLoader will also check to see if
your source code has changed, and re-compile it as necessary!
For
example, if I wanted to compile and load a HelloWorld.java (that has a
simple 'Hello World' method) in my ./src directory, I could do:
No more ANT tasks, no more export to .jars, JavaLoader handles the compilation and deployment of your Java code for you.
ColdFusion Component Dynamic Proxy
For those of you who are not aware of what a Dynamic Proxy
in Java is, it is essentially an Object that can mimic an object that
implements a given set of Interfaces. The method invocations against
that object are intercepted, and you can essentially do whatever you
like with them.
In JavaLoader 1.0, I've written a Dynamic Proxy that you are able to wrap around a CFC, and thereby make Java Objects think
they are interacting with a native Java object, but are in fact,
talking to your CFC. When a Java Object calls a method on the Dynamic
Proxy, it is passed through and invoke directly on the CFC transparent
to your Java layer.
For example, the java.lang.Thread Java object can take an instance of the interface java.lang.Runnable in its constructor argument. Using JavaLoader's Dynamic Proxy, we can pass it what it thinks is an instance of the Runnable Interface, but is in actuality a ColdFusion Component like so:
//give us access to the CFCDynamicProxy
CFCDynamicProxy = javaloaderloader.create("com.compoundtheory.coldfusion.cfc.CFCDynamicProxy");
//create my Runner CFC
myRunner = createObject("component", "MyRunner").init();
//wrap it in a Dynamic Proxy, that implements Runnable
runnerProxy = CFCDynamicProxy.createInstance(myRunner, ["java.lang.Runnable"]);
//pass it to the Thread and call run()
thread = createObject("java", "java.lang.Thread").init(runnerProxy);
thread.run();
...and suddenly we have true seamless communication from Java to ColdFusion components.
This
is actually something I wish I had written three years ago. It would
have radically altered how I would have written a lot of software.
Spring Integration
A natural progression from the Dynamic Proxy, was to implement a Custom Schema in Spring, so that you could instantiate and dependency inject your ColdFusion components with your Java Objects inside Spring itself.
Initialising
Spring with JavaLoader is covered in the documentation, but to give you
a quick taste, once we have Spring running, we can include a CFC in our
spring.xml with:
Where script-source is the absolute path to the CFC you want to instantiate, and script-interfaces are the Java interfaces that your dynamic proxy will implement, and your CFC has mirrored inside it.
This
Component then gets treated like a normal Spring bean, so it can be
autowired, injected, aop'd and all the usual functionality that a
Spring bean has access to.
There are some downsides to implementing Components inside Spring, so be sure to check out the documentation for more details.
Since
this has become a far larger project than its original inception, I've
done away with the Riaforge Forum, and have started a new javaloader-dev google group for support and discussion of JavaLoader and Java and ColdFusion integration.
While
I'm very confident in the code that I have written for JavaLoader 1.0,
I decided to call it an Alpha, simply because it has only really been
tested by me and my Unit Tests. Knowing from experience that users of
Open Source Software have a tendency to use and abuse libraries far
above and beyond what the author had ever originally dreamed led me to
decide to call it Alpha until the community has really had a good
change to play with it.
In the previous twoarticles, we started the discussion of Hibernate Sessions and how they related to the ColdFusion 9 ORM. We noted the fact that a Hibernate Session
keeps track of what objects it loads, and can therefore update it when
it realises that the object has changed. The question then posses
itself - if a Hibernate Session has closed, which happens at the end of every ColdFusion request, what happens to the objects that it was managing?
Going forward, I'm going to reuse the Musician model I used in the Introducing ORM in Adobe ColdFusion 9 beta Adobe DevNet article, for which the updated code samples can be found here.
The very short version for those who don't wish to look at the code, is
a model in which we have a Musician, who has a name, and an age, and
he/she also has a many-to-many relationship to an array of Instruments,
which they can play.
As far as Hibernate is concerned, there are three different states for any Object that it interacts with.
Transient
An Object/Entity (you could use either word) is Transient, if it has just been created with a createObject() or new operator, but has yet to be saved with the EntitySave() method.
Obviously, if transient objects are not persisted via EntitySave(),
then they never get stored in the database, and will eventually be
garbage collected.
For example, our musician john below is a transient object:
<cfscript> import com.*; //this is transient john = new Musician("Transient John"); </cfscript>
Since john is never saved, it will always be transient.
Persistent
A Persistent object is one that has either come from, or has been saved in the database, within the scope of the current Hibernate Session (which is an important point to note).
For example, we can make our musician john below persistent, by:
<cfscript> import com.*; //join is transient john = new Musician("Transient John"); john.setAge(47);
//john is now persistent EntitySave(john); </cfscript>
Therefore, since john was inserted into the database, he is persistet.
By the same token, if we retrieve an object through the ORM, it is also persistent:
<cfscript> //larry's ID is 2
//larry is also persistent larry = EntityLoad("Musician", 2, true); //As larry is persistent, we can change his age, which will be reflected in the database larry.setAge(19); </cfscript>
Since larry was retrieved from the database, he is also persistent.
Detached
Detached objects are ones that used to be persistent, but the Hibernate Session has
closed. Detached objects were one of the biggest things for me to wrap
my head around when starting with Hibernate, and can cause a few really
tricky errors to occur.
To explain detached a little better, we can go through a
scenario that will causes them, as I think this is something that a lot
of ColdFusion developers are going to end up running in to.
Let's have a new musician, called Tim, and time is going to have 2 instruments that he plays, the Flute and Piano.
We are going to grab Tim out of the database, and then put him in our ColdFusion Session scope, so we can keep track of him from request to request.
<cfscript> tim = EntityLoad("Musician", 2, true); session.musician = tim; </cfscript>
For this request, Tim will be persistent, however, on the next request, Tim is now detached, as the Hibernate Session will have closed at the end of the previous request.
You will see the following (not well written) error:
failed to lazily initialize a collection of role: Musician.instruments, no session or session was closed
If you are running ColdFusion via the console, you will also see something similar to:
org.hibernate.LazyInitializationException: failed to lazily initialize
a collection of role: Musician.instruments, no session or session was
closed
What does this error even mean?
First of all, we should remember that the collection of Instruments our Musician can play are lazy loaded. This means they aren't actually loaded from the database into the object until you actually request them.
The important thing to note here, is that lazy loading with Hibernate can only occur if the Hibernate Session that the object was originally loaded from is still open. To put it another way, if the Hibernate Session has closed, then Hibernate has no record of what the object was doing, and therefore can't do any lazy loading.
In the above example, since the previous ColdFusion request ended, the Hibernate Session has closed, and if you attempt to access a relationship that is set to lazy load, and has yet to be loaded, this error occurs.
Its should be noted that singular properties are not lazy loaded -
therefore, if you are just grabbing a property on an object that is
detached, no error will be thrown.
What can we do to mitigate this issue? There are a few options, one of
which is, to simply retrieve the object all over again, like so:
As then the object we are dealing with is considered persistent, as it is attached to the currently open Hibernate Session. This tends to be the easiest of the options when dealing with detached objects.
Another interesting way of making our detached object persistent is like so:
This may look strange, but what is actually happening when you call EntitySave(), is that it calls the Hibernate method saveOrUpdate().
A quick version of what saveOrUpdate() does is -
If the object being passed in is transient, INSERT it to the database.
If the object is already persistent in the current Hibernate Session, do nothing.
If there is already an object that represents the same Entity data persistent in the Session, throw an exception
Otherwise, update the database with the information in the passed in object, and set its state to persistent.
Therefore in the example above, the data stored in session.musician will be saved again to the database, and since session.musician is now deemed persistent, we can successfully do lazy loading of the instruments.
a different object with the same identifier value was already associated with the session: [Musician#2]
Since Hibernate deems the persistent object that is currently connected to the Hibernate Session much more important than the detached object, which makes sense, as the persistent object has the most up to date version of the data stored in the database.
Generally speaking, if you at all can, try and avoid having detached
objects in your application, as this is generally a Good Thing™, and
will allow you to avoid some pretty decent headaches as you develop
with CF9 ORM. Strategies such as storing IDs in the ColdFusion Session
and having an API to return a new Object by that ID on request is often
a good way to go, rather than storing the Entity itself in the ColdFusion Session. That being said, if your architecture means that you have to have detached objects this series should be enough to give you a basic grounding the concepts of
object state, and how to manage it.
If you want to learn
more about managing Hibernate Object States (and there is far more to it than I have written here), you can read more about this in the Hibernate documentation,
but this should hopefully have given you a decent enough overview to
avoid some possible pitfalls when handling persistence with ColdFusion
9's ORM integration.
This year it is going to be my absolute pleasure to be attending and presenting for the first time ever at the Adobe MAX conference.
My presentation is entitled ColdFusion For Java Developers and is scheduled for Wednesday, 5:00pm.
Primarily
aimed at Java developers, it is going to cover a myriad of ways that
you can integrate your already existing Java code base into a
ColdFusion application, and also get your Java model talking seamlessly
to your ColdFusion code base.
The official synopsis is:
Ever since ColdFusion was re-implemented as a J2EE application, the
benefits of combining Java and ColdFusion application development have
been easy to see, and widely used across the ColdFusion landscape.
This
talk will look at some of the ways we can already take advantage of
common Java libraries in ColdFusion, and enable Java developers to
leverage the libraries and frameworks that they are already using. We will also be looking at
how Java developers can seamlessly leverage the dynamic scripting
language of ColdFusion, within their Java code, to easily enable them
to take advantage of some of the RAD features that come bundled with
ColdFusion.
That
being said, if you are a ColdFusion developer who likes to also work
with Java, you may wish to come along, as there will be several
techniques demonstrated that will be provided by the forthcoming JavaLoader 1.0 release that should (hopefully) interest you greatly.
Now that ColdFusion 9 is in public beta, I wanted to write some
articles explaining how Hibernate is actually working under the hood of
the nice ORM integration we now have. Like all powerful tools, they
can do a great many things - but only if you really understand how they
work. Otherwise, they are potential for disaster if you don't know
what you are doing.
Hibernate Sessions
The first thing we need to cover is the concept of Hibernate Sessions,
which has been nicely hidden away from ColdFusion developers by the
Adobe Engineering team, but is still something that you need to really
understand to be able to leverage Hibernate in CF9 properly. (At least
IMHO).
First things first - don't confuse this with ColdFusion Sessions, or
any sort of browser session. Hibernate is built as a generic solution
to ORM, so it has no specific ties to web state. A Hibernate Session
is a lightweight object that is used to delineate when Hibernate is
meant to start working, and when it is meant to stop, which is often
referred to as a unit of work. This may span several transactions and/or INSERT, UPDATE, DELETE and SELECT statements as the Session gets used.
This is very much like the start and stop of a conversation, we say
hello at the beginning, and say goodbye at the end, and whatever we
need to discuss goes on in between, except we just tell the Session to open() and close(), rather than "hello" and "goodbye".
Actual Hibernate Java code would look something like this:
//open the session from the SessionFactory (which is where sessions come from)
Session session = sessionFactory.openSession();
//get fred, our caveman
Caveman fred = session.get("Caveman", 2);
//change his age to 32
fred.age(32);
//close the session, which will update fred.
session.close();
I'm not going to get into where the SessionFactory comes from, as all you really need to know is: it's where Hibernate Sessions come from.
So what we are doing here is:
opening a Session, i.e. we tell Hibernate 'Hi', which starts our conversation.
Tell Hibernate to retrieve our Caveman fred, through our session.
We change the age of fred to 32.
We close our session, and say 'GoodBye' to Hibernate for now.
There are several things that happen here that are worth noting at this point, even in this relatively simple piece of code.
The first is, by doing session.get("Caveman", 2), we have effectively told our Session to track this object, and make note of any changes that happen to it.
We have also cached a copy of fred in the Session itself, so that if we were to attempt to retrieve fred again, we would get the same copy. This is both for performance, and also so that the Hibernate Session only has to deal with one object that represents the data behind fred, which will be a fairly important concept as we continue in this article series.
Since our Session is tracking fred, any change we make to fred will be persisted to the database when we close our Session. In that way, Hibernate is quite transparent in the way it implements persistence.
It should also be noted at this point: Unless you specify otherwise,
Hibernate will only run INSERT, UPDATE and DELETE when the session is
closed, generally as a batch. This is so that, for example, multiple
changes happen to fred during the length of our Session,
there only needs to be one UPDATE statement when we are finished. This
is an important thing to note, as it is possible to get confused as to
why your changes to your objects aren't appearing in your database during your Session, but do appear afterwards.
Hibernate Sessions and Web Applications
In web applications, it is usual to manage Hibernate Sessions by
opening one up at the beginning of a HTTP request, and then closing at
the end of the request. The length of a single HTTP request is a good
size to carry on a nice conversation with Hibernate.
Strangely enough, this is exactly what ColdFusion is doing for you behind the scenes. It starts a Hibernate Session
when your request starts, and ties it to that particular request, and
then closes it at the end of the request. (It is quite likely that the
Session actually only gets created when first requested, but for the
sake of argument it is easier to explain it as it gets opened at the
start of the request)
From there, each of the ORM functions, EntityLoad, EntitySave, EntityDelete, etc, all interact with that Session that exists behind the scenes, for that request.
This means that changes to your objects will also not be persisted to your database, until the end of your ColdFusion request, as that is when the Hibernate Session is closed. In later articles, I'm going to cover some ways this can cause some trouble, and how to work around them.
You are actually able to flush the Session, which
forces it to persist any changes to the database right then and there,
and not wait until the end of the request, which can sometimes be very
useful.
To do this in Java, you could go:
session.flush();
But in ColdFusion you write:
ORMFlush();
UPDATE: You can read more about Session Flushing here, which outlines that there are times in which Hibernate will flush a Session in the middle of Session, however, there is often no guarantee except in specific circumstances.
It's also interesting to note you have direct access to the Hibernate Session if you so desire through ORMGetSession(), which gives you access to that request's Hibernate Session object, so you can interact with it directly.
There is much more to Hibernate Sessions and how they relate to the
objects that they manage, but this will give us a good start to build
upon for now.
While I am recovering from the Australian WebDU conference, a few days before that started I got off a plane after the end of the wonderful cf.Objective() conference.
I have to say, this year's cfObjective() was the best organised out of all the years I have been to. As per usual, the content was stellar, the hotel was lovely, and it was an absolute pleasure to catch up with everyone at the conference. I have to give a big 'congratulations' to Jared, Steven, Jim and the rest of the cf.Objective() crew for putting together such a smooth and professional conference.
I had the pleasure of doing two sessions, Rapid OO Development with ColdFusion Frameworks, which covered a variety of techniques on how to increase your development speed when building OO models, and I was very happy to see that it seemed to have been very well received. I had one attendee let me know that 'Now I know why I'm using ColdSpring! I was using it before, but now I know why', which is an amazing thing to hear as a presenter, that you've managed to create an 'Aha!' moment for someone.
I also did my Introduction to Building Applications with Transfer ORM, which was a repeat of the session that I did last year. Unfortunately Ray Camden couldn't make it to do his Transfer session, so I was called in at the last minute to take his place.
The big news that we announced at cf.Objective(), is that I will now be the lead developer on the ColdSpring project. Since Chris Scott's major focus these days is the Swiz Flex framework, he decided it was time to pass on the reigns, and since I tend to talk to him regularly about Cold/Spring, have contributed code to ColdSpring , and know about running an Open Source project, he seemed to think I would be a good fit. I'm pretty excited about the opportunity, and have discussed some great ideas with theColdSpring development group, of which Chris is going to stay on as lead architect. I expect we will start off by building the infrastructure around the project, e.g. a centralised wiki, ticket tracker etc, and then move on to some more interesting items.
The obvious question there is, of course, what does this mean for Transfer? (I think I need to start writing down how many times I've been asked if it's 'Dead'. Does anyone actually expect a 'yes' for an answer?), and quite frankly, I don't see this impacting on Transfer much at all, simply because this is going to be code that I would have probably ended up writing on ColdSpring anyway, but it is now a more formalised relationship. When I run into a feature or a bug on an Open Source project, that I want to be implemented, my first natural reaction is to start looking into the code, and writing the feature. This was first exemplified by my contribution toColdSpring of annotation based pointcuts. There are several aspects of ColdSpring I wanted to improve on, so it was just a natural reaction for me to end up writing code for it.
As stated, the content at cf.Objective() was brilliant as per usual, with my own personal highlights being, Advanced ColdFusion Server Administration (Adam Lehman), Advanced ColdFusion 9 ORM (Terry Ryan) and ColdFusion Portlets (Adam Haskell).
Thinking about the content, I have a little confession to make, that I realised on the way back from cf.Objective() this year. I have a tendency to go to the wrong sessions when at a conference. This may sound like a weird thing, but I realised the last few years I tend to go to sessions that I already know a lot about, just to see if they say something a little bit extra that I can add to my knowledge base. Quite often I end up walking out feeling like I haven't added much to my repertoire. Really what I should be doing is going (mostly) to sessions in which I know absolutely nothing about, which means I actually get the best return on the my investment in the conference. While it may not be specifically applicable to what I'm currently doing, at the very least it will inspire me to do some interesting new things, and may give me some knowledge that I can then apply at some point in the future. This is a philosophy I plan on applying to all future conferences that I attend.
Finally, I also had the opportunity to be part of a CFConversations round-table on the second night of the conference. Brian Meloche, Andy Powell, Andy Matthews and I had a really good chat about the conference in general, our thoughts on some of Adobe's upcoming products, various other topics relating to ColdFusion. It was lots of fun to do, and you can download and/or read more about it here.
Again, thanks to all the cf.Objective() crew, and look forward to seeing many of you again at cf.Objective(ANZ).
Conduit requires that you set up BlazeDS
on your machine, as well as install the conduit.jar file, and some CFCs
as well. I'm not going to go into too much details here, as you can
read about in the Library Installation section of the documentation.
To configure Conduit, we open up our remoting-config.xml, and we add the Conduit adapter to the <adapters> section:
And now we continue by adding a new Destination to our remoting-config.xml, which we will then call from Flex via RemoteObject (examples of the configuration are provided in the downloads).
<destination id="Conduit"> <channels> <channel ref="my-cfamf" /> </channels> <adapter ref="conduit" /> <properties> <source>*</source> <cfcs> <!--
Whether or not reload the CFCs below on each request.
Useful for debugging when building new invokers,serialisers
or deserialisers
--> <reloadcfcs>false</reloadcfcs> <!--
The CFC that invokes the remote method call
--> <invoker>conduit.core.CFCInvoker</invoker> <!--
Translates CF=>AS3
--> <serialiser>conduit.core.CFSerialiser</serialiser> <!--
Translates AS3=>CF
--> <deserialiser>conduit.core.CFDeserialiser</deserialiser> </cfcs> </properties> </destination>
The id of the destination can be whatever you like, but we started with
'Conduit', so we would know what we are calling from Flex.
The <adapter> is specified to use the conduit adapter, rather than the default cf-object adapter.
We then set a series of properties, most important of which are setup
within the <cfc> section. This section controls what CFCs are
called upon to perform various duties within the ColdFusion <=>
Flex communication process.
Just to emphasise this point - this means that the majority of the heavy lifting done by Conduit is done with ColdFusion code.
This makes it really easy to extend, change, manipulate or debug. It
gives you almost complete control over the AS3<=>CF translation
process, without you having to know much about Java at all (I will
admit there are some Java classes involved).
We can see from there are three CFCs that the Conduit adapter has configured for it.
The <invoker>
This is the CFC that does the actual method calling. In the instance
of the conduit.core.CFCInvoker that comes with Conduit, all it does is
create an instance of the CFC that has been requested in the
<RemoteObject>'s source attribute, and calls the passed with method name and any parameters that were passed down from Flex.
The code looks something like this (just to show you how simple it is):
<cffunction name="execute" hint="Creates a cfc, and invokes a methodon it" access="public" returntype="any" output="false"> <cfargument name="source" hint="the cfc source" type="string" required="Yes"> <cfargument name="methodName" hint="the name of the method" type="string" required="Yes"> <cfargument name="params" hint="the parameters to pass in" type="any" required="Yes"> <cfscript> var local = {}; var object = createObject("component", "#arguments.source#"); </cfscript>
The job of this CFC is to take whatever is returned from the
<invoker> and translate it into whatever you want to return back
to Flex. Since BlazeDS handles the AMF conversion part of it for us,
the conduit.core.CFSerialiser only really needs to convert CFCs in Actionscript objects, set all their properties correct, and we are good to go.
The <deserialiser>
The deserialiser CFC's job it to take the incoming parameters that come
down from a Flex RemoteObject call, and translate them into something
usable for the <serialiser> CFC.
Again, since BlazeDS does a lot of the heavy lifting, the
conduit.core.CFDeseriailser's main job, is to convert Actionscript
Objects into their appropriate CFCs, and set all their properties
correctly.
Changing how things work
While there are plans to put some interesting enhancements for Conduit
to provide above and beyond what ColdFusion remoting already does, the
real power of Conduit comes from being able to write your own custom
Invoker, Serialiser or Deserialiser.
Say for example, when we send information from ColdFusion to Flex, we
want to reverse every Simple value (String, date, numeric) that we come
across. Don't ask me why you would want to do this ;o), maybe you just like messing with your co-workers.
First thing we need to do is write our own custom CFSerialiser.cfc. For this example, I'm going to reuse the conduit folder I would have setup, which already has a /conduit ColdFusion mapping pointing to it, and create a new folder called reverse inside it.
So I create a new component, under /conduit/reverse, and call in
CFReverseSerialiser.cfc, and make it extend conduit.core.CFSerialiser.
We will now overwrite the translate method, which handles what data gets converted, and how, depending on its data type.
The code would look something like this:
<cfcomponent output="false" extends="conduit.core.CFSeriaiser" hint="Component for serialising CF=>AS3, in reverse">
<cffunction name="translate" hint="translation function for objects,reverses simple values" access="private" returntype="any"output="false"> <cfargument name="object" hint="the object to serialise" type="any" required="no"> <cfargument name="cache" hint="local reference cache for cyclic graphs" type="any" required="Yes"> <cfscript> if(isSimpleValue(arguments.object) { >//if it's simple, then reverse it! return reverse(arguments.object); } else { return super.translate(argumentCollection=arguments); } </cfscript> </cffunction> </cfcomponent>
And we can configure a special 'ConduitReverse' destination for anyone who wants reversed Strings in their code: <destination id="ConduitReverse"> <channels> <channel ref="my-cfamf" /> </channels> <adapter ref="conduit" /> <properties> <source>*</source> <cfcs> <reloadcfcs>false</reloadcfcs> <invoker>conduit.core.CFCInvoker</invoker> <serialiser>conduit.reverse.CFReverseSerialiser</serialiser> <deserialiser>conduit.core.CFDeserialiser</deserialiser> </cfcs> </properties> </destination>
As you can see, we can do almost anything we want to the communication
process, simply by extending the core components (or even writing brand
new ones) and because its at a low level, its almost completely
seamless to those who are writing the ColdFusion and/or Flex code. That
being said, its open source, so we can add extra logging and/or
debugging as we need. No more failing silently!
Trying it out
This is still Alpha code, and there is a lot of logging
currently in it, but there is enough there for you to start playing.
Code is in SVN, and daily builds are available from the Google Group (saves you having to compile the .jar file). The documentation is slowly getting fleshed out, but there is enough there to get started.
I am more than happy to get code contributions, and/or ideas for how the to expand on the current ColdFusion remoting feature so send through whatever you have.
As per usual, this is something that came out of my work with Transfer, but is something that applies to any ColdFusion application that exists.
So in the context of ColdFusion, what exactly are we referring to when we say Thread? Generally the first thing we think of is <cfthread>,
which executes some code on its own given Thread. But, we should also
remember that the original page that was executing, is its own Thread
as well. If we run a scheduled task, that is also it's own Thread.
"A thread in computer science is short for a thread of execution. Threads are a way for a program to split itself into two or more simultaneously (or pseudo-simultaneously) running tasks..."
So when looking at Threads, we can consider:
Any Coldfusion page execution,
Any remote CFC execution
Any Scheduled Task execution
Any CFThread execution
To be its own Thread, because, it is!
So what defines a Selfish Thread?
A Selfish Thread is a thread that takes up almost all of the CPU's processing, without allowing any other Thread to be able to utilise the CPU at all.
Some code like this, would be a good example of a Thread being selfish -
It's a very tight loop, and there is no waiting, or pausing, or 'room'
for any other processing to do anything else while this loop processes.
Now it should be noted, that a Selfish Thread may not necessarily be a
bad thing. In many instances, we want this loop to completed, without
waiting for any other Thread to interrupt it. But in cases when Thread
execution can go on for a long time, this can be highly disruptive to
an application, as nothing else can be done during that time.
The common CF solution I often see for this, is the scheduled task that
runs at 3:00am, so that it doesn't bother any of the users. This can
work perfectly well for many applications, but what if your application
is 24 hours? Or is something that has to run every hour, what do you do
then?
Before we get into this too much, I want to make note of something - managing Threads is bit black magic, and a bit trial and error. Since Threads are managed differently per OS, and there are differences per JVM, some of these techniques will work, and some will not, so make sure you test everything thoroughly so you know that it is affective for your OS and JVM configuration.
The other thing to note, is that any Thread that is running, is an actual instance of th Java object java.lang.Thread. If at any point and time we want access to the actual Thread object that the given process is running we can run:
currentThread = createObject("java", "java.lang.Thread").currentThread();
Will return a reference to the currently executing Thread object, which will be very handy as we move along.
The first thing we should look at, is <cfthread>. CFThread has a 'priority' attribute that can be set to 'HIGH', 'NORMAL' or 'LOW', which should control
the level of priority that a Thread has. For example, a HIGH priority
thread should have processing precedence over a LOW priority thread.
For example:
<cfthread action="run" name="foo1" priority="LOW"> <!--- do some processing ---> </cfthread>
In reality, I've not seen this actually do much (in my tests), and it does not seem to actually effect a Thread's Thread.getPriority(),
which we will talk about later. That being said, there may be some
other mechanism under the hood, and its not going to hurt anything if
you choose to use it.
From here, we can look at setting a Threads priority, which can be
applied to any CF based Thread (i.e. pages, scheduled tasks, cfthread
etc). A Threads priority goes from 1 to 10, where 1 is the lowest
priority, and 10 is the highest. 5 is usually considered 'Normal'.
In theory, a lower priority Thread should give way to a high priority
Thread whenever the higher priority Thread requires CPU processing
time. As stated earlier, depending onJVM and OS, this may, or may not happen.
To set the Thread's priority, all you need to do is grab the current thread, like we did above and call:
currentThread.setPriority(2); //set it to a lower priority.
//do some processing...
Since ColdFusion tends to pool Thread (i.e. stores them for reuse), we
should reset the Thread's priority after we are done with it, so that
it doesn't stay that when it gets used to execute another piece of
code. e.g.
priority = currentThread.getPriority();
currentThread.setPriority(2); //set it to a lower priority.
//do some expensive, long running processing...
currentThread.setPriority(priority); //reset it
This way, when the Thread get re-used, the Priority is not set to
something that is inappropriate for the processing it is doing.
There are also mechanisms in Java that allow you tell the JVM when a good time is for the current thread to yield to other threads that need to do some processing.
This simply hints to the JVM that 'hey! now would be a really good time for me to pause for a second, if you wanted to do something else'. The JVM can totally ignore this if it chooses, and depending on OS and JVM, it may well do.
To do this, we call the static method yield(), on java.lang.Thread, like so:
createObject("java", "java.lang.Thread").yield();
So we can now take our very selfish loop above, and do something similar to:
<cfscript> for(counter = 1; counter <= 10000; counter++) { writeOutPut(counter &"<br/>"); createObject("java", "java.lang.Thread").yield(); //here is a good place to pause } </cfscript>
This is actually a very poor use of yield(), simply because in a
display, we would never want the server to pause when displaying some
data, but it displays how it works reasonably well.
The interesting thing is, yield() automatically resolves what the
current thread it is processing on, and works that way, rather than the
setPriority() method we saw above, which required us to use a specific Thread.
Quite probably the least useful, but the most consistent way of managing selfish threads, is by putting the thread to sleep, which will allow other Threads access to the CPU while that thread is asleep.
This is the least useful, as no matter what, the Thread
will pause. Nothing else may be happening on the server, but the
Thread will pause anyway, which can mean wasted cycles for whatever it
is you are doing.
That being said, this will always work, no matter what JVM or OS you are on, so there is a trade off.
There are three ways we can make the current thread sleep in ColdFusion,
Either way, in the above example, the current thread will pause for 1000 milliseconds.
In a real world example, there is no reason we can't combine these
techniques. If we had some processing we wanted to happen
asynchronously, but we knew it was going to take a while to complete,
we could do something like the following:
When:
17th of July, Meeting starts at 7:00, so get there before hand (doors open at 6:30).
Agenda:
Elliott Sprehn
Elliott is a software engineer at TeraTech, Inc. and the lead developer for the Conferences department. He has worked on a variety
of projects that include network services, realtime messaging systems, survey processors and most commonly web applications. As a strong advocate of web standards and accessibility he volunteers his free time to help new developers learn standards and best practices. He is also proponent of design patterns and draws his vision about application design from experience with many languages that include ColdFusion, Java, ruby and php.
Internals of the Adobe ColdFusion Server
To many CF developers the ColdFusion server is a block box that just works. This presentation takes a look at how the internals of the engine operate. The presentation covers how variables, scopes, functions, cfcs, custom tags, includes and java objects are implemented and how we can use this knowledge to do things not normally possible with the CF functions.
We'll look at how to implement several features in CF6, CF7 and CF8 like:
- Query Caching with the native CF query cache for queries that use
cfqueryparam.
- Create and execute queries that use cfqueryparam from cfscript.
- Create a function like CF8 to get database info for a datasource.
- Add global mappings by appending to a structure like this.mappings in CF8.
- Call functions and component methods and inspect the local scope
after their execution.
- Access the Application.cfc anywhere in an application.
- Allow passing arrays of values in the url scope instead of lists
when there are duplicate keys.
If you are going to attend, please RSVP to mark (dot) mandel (at) gmail (dot) com.
Only those that RSVP are eligible for the door prizes, so make sure you apply!
Not a huge amount to report. This release simply re-orders some of the class loading, so it is child first.
This
is important in case you load up a newer (or older) version of a
library that ColdFusion already has access to, and you want to make
sure you get the library you want, not the library loaded by ColdFusion.
That and I fixed a bug in the JavaProxy.cfc that was sent to me a loooong time ago, that will fix an issue with resolving overloaded Java Methods.
We have a reasonably large application that we have been building for the past 2 years or so, with MachII, ColdSpring and Transfer, and we moved over to ColdFusion 8, pretty much as soon as the Beta came out, and have been developing with it ever since.
To give you an idea of the size of the project, some metrics for you, we have 1064 CFCs, 329 .cfm pages, 161 configured Transfer Objects, 40 MachII
frameworks instances (we started this before modules), and 201 Tables.
This actually results in about 22,000 .class files being generated by
ColdFusion behind the scenes.
What we began to experience is
extremely long start up times when the server was first started up. For
example with no class files saved on the server, we have a start time
of around 20 minutes before the application was responsive, and if
multiple people hit the site at once, we were looking at around 45
minutes before the application would perform usably.
With class
files saved, this dropped down to 9 minutes, but we still ran into huge
difficulties with multiple users hitting the site at once. On top of
that, whenever we had to upload a change, clearing the template cache
so the change would propagate brought the server to it's knees, so we
were forced to restart CF with every deployment of code.
So
after much haranguing, and talking to a whole slew of people, both
Adobe, and non Adobe (you all know who you are, and thank you very
much for the time and effort you all put into helping out on this), it
eventually got narrowed down to a bottleneck in Java 6 as described here (as also reported by Sean Corfield )
So
for those of you not so familiar with Java, what does all that actually
mean? Well, let's look at what ColdFusion does behind the scenes when
you run a .cfm or .cfc page:
Checks to see if it has the Java Classes already in memory to do what you have requested - if not,
CF Reads in the file you are executing
Parses the CFML
Converts the parsed CFML data structure, and converts it into Java code
Compiled that Java code down to actual Java .class files
Loads the resulting .class file into memory
Executes the Java code contained in the .class file as necessary.
So
probably nothing too revolutionary in there in terms of our
understanding of the ColdFusion process, however, there is a big bottle
neck in Java 6 where the loading into memory Java classes is really really really slow (maybe I need another really there), so step 1.5 on the above processes takes a very long time to get through - and the problem only gets worse
when there are large numbers of very small classes - which in almost
any ColdFusion application, and ours in particular, there are.
There are several places in which this issue can cause serious problems -
At System Start Up
As discussed previously.
Development.
Obviously during development, files tend to change quite regularly.
During development of our application, it would not be strange for me
to be spending several minutes, to the tens of minutes waiting for a small change in either a .cfm or .cfc to come through. This simply slows down the amount of work that you are able to do in any given time frame.
At Run Time
Since ColdFusion has a finite limit on the number of cached templates
(which are just the .class files mentioned earlier stored in memory),
it is quite likely that at some point during an application life cycle,
part of what is stored in the template cache will get purged so as to
make room for other Java classes that have become active. We hit the
same issue as we hit on step 1.5 as above, as the ColdFusion server
slows down as it pulls in the required Java classes into memory. This
can result in random slowdowns in the application, which are hard to
reproduce.
This becomes a larger issue on shared host systems, in which a trusted
template cache is impossible, and it is quite likely the ColdFusion
server is constantly moving the generated Java code in and out of it's
template cache, as multiple systems require different ColdFusion code
to be executed.
It wasn't until we had one of those head-slap moments when a co-worker
turned around to me and said 'well.. why don't we try Java 1.5?'?
Once installed, suddenly everything started working like we wanted it to, performance wise.
Since everyone loves a pretty graph, here are some metrics on server
start up, taking a Selenium script through a series of steps through
one part of our application, so you can see the considerable difference
between the two Java Versions
Not only has this made our production systems run incredibly fast, it
also means that we are able to upload code to the production server,
clear the template cache, clear the application cache, and we are good
to go, there is no need to restart ColdFusion for changes.
This also means I can develop without having to take a coffee break in
between code changes, which has upped my productivity as well, in fact,
I can now develop happily in Machii with the config mode set to
constantly reload, and performance is no problem at all.
As far as I am aware, this issue does not exist in the development
snapshots of Java 7, and apparently a fix is in the pipeline for 1.6,
but Sun hasn't been forthcoming about the date.
Until that time, I would suggest moving your ColdFusion 8 servers over to Java 1.5, and enjoy the speed improvements!!!
It pretty much does exactly the same thing that the previous versions
did,
however, it now works in ColdFusion 8, when the setting 'Disable access
to internal ColdFusion Java components' is turned on, as it uses its
own JavaProxy CFC, which may often be turned on on shared hosts.
That's really about it, nothing more interesting than that.
One restriction I ran into today when doing some ColdFusion8 specific
performance enhancements to
Transfer
with <cfthread> is that you can't make a child thread within an already
created <cfthread>, so something like:
Will end up with lots of errors in your logs saying things like:
Error","cfthread-1","09/24/07","12:03:42",,"A: Thread B cannot be created. Child
threads are not supported."
Which is probably not what you actually want, as you would most likely like for
you code to actually execute.
I'm not a huge fan of this limitation, but I can kind of understand where the
engineers are coming from - they don't want us CF developers to shoot ourselves
in the foot with ridiculously spawning multi threaded applications.
So, in my current situation, I wasn't too fussed if I had a whole, brand new
thread, but where my code was being run, I was so far down the stack of method
calls, I have no way of knowing if the original call had come from a standard
ColdFusion thread, or if it had come from a <cfthread> create thread, and
both of which were highly possible. However, if the execution wasn't coming from a <cfthread> created
thread, I wanted to execute my code inside a <cfthread> statement, as I
didn't care what the results where, and it would be faster for the end
user just to have the code run asynchronously.
If that just made your head hurt, I basically have a series of method calls that
look like:
A()
- makes a <cfthread> call, and calls C()
B()
- Does some work...
- Calls C() from the current thread.
C()
- makes a <cfthread> call, and then does some work
This is an over simplification of the process (mine is far more levels deep),
but gives you an idea of the trouble I was facing - the method call could go
through A() or through B(), and at the level of C() I wasn't aware of whether or
not I had started on the base ColdFusion thread or not.
I could have done this passing a parameter all the way down the method chain to
tell it if it should fire asynchronously, but that seemed like a lot of extra
work, and would have put a level of complexity within the whole process, and any
other process that was related to this, that it didn't seem like the appropriate
course of action.
So I thought to myself - there has to be a way to find out if the current thread
is a regular, ColdFusion thread, or if it is a <cfthread> invoked
thread, and funnily enough, skipping into some Java, there is!
The first thing we have to do, is get access to the
java.lang.Thread
object:
Now, the Thread object, has a great method
'currentThread()',
which returns the current thread that the executing code is on:
currentThread = Thread.currentThread();
What is good to know, is that Threads in Java can exist in
ThreadGroups,
which is a handy way of grouping Threads together for common operations, and
other clever things. The nice thing here, is that ThreadGroups have a
method called
'getName()'
which returns the name of the ThreadGroup.
Strangely enough, <cfthread> created threads live inside the ThreadGroup
named 'cfthread'! So now we can compare this against the name of the current
Thread's ThreadGroup
We can also wrap this up nicely into a quick little UDF:
<cffunction name="amInCFThread" hint="returns 'true' if the current thread is a cfthread, returns false otherwise" access="public"returntype="boolean" output="false"> <cfscript> var Thread = createObject("java","java.lang.Thread");
First of all, you may be wondering 'what on earth is a JavaProxy?', well,
to answer that question, it is the Java class that does all the work behind the
scenes in ColdFusion to allow you to be able to write all that Java code in-line
in your ColdFusion CFCs and CFM pages by taking the Coldfusion invocations you
have implemented, and passed them to the native Java objects that you want to
use.
To further your understanding, if you are at all interested, you can also read
up on the
proxy
design pattern here.
Now, what some people may or may not realise, is that inside
JavaLoader,
I create an instance of the coldfusion.runtime.java.JavaProxy class, so it
becomes really easy for developers to create and use instances of Java objects
that are loaded from external .jar files within their applications. I have
a good blog post on doing this
here.
Now just the other day, I became aware of a new setting in ColdFusion 8 entitled
'Disable Access to internal ColdFusion Java components', that really threw me
for a bend.
For people who run shared hosts, they probably think of this as a g-d send, in
that it will disable access to coldfusion.runtime.ServiceFactory - and for that,
I totally understand, however, it completely locks down access to any Java
object that sits under the coldfusion.* package space.
What does this mean? It means that JavaLoader, no longer works, along with
anyotherprojectthat also
uses JavaLoader could quite potentially not work on some shared hosts providers
that have upgraded to ColdFusion 8!
Why did Adobe decided to do this? Not so sure! However, with the power that we have
in ColdFusion 8, we are able to implement our own JavaProxy, that should be able
to be seamlessly interchanged with the ColdFusion native JavaProxy!
(Disclaimer: This is the first run at this code, and it works in the given tests
I have tried on it. I will be running it against all the unit tests on
Transfer, and when they all work perfectly, and Transfer can run with this CF8
restriction in place, I will release the full code as part of a new version of
JavaLoader)
There are two things that allow us to do this -
Nothing stopping us from using Java Reflection to dynamically call methods
on a Java Class or Instance.
In CF8 we got 'onMissingMethod()' - so we have a hook into every method that
is fired on a CFC, regardless of whether or not it has been implemented.
For those of you who aren't that familiar with the term
reflection - it essentially means that we
are able to introspect a Java Class or Object, determine what methods and/or
properties it has, and dynamically call them at run-time. If you want to
read more, the Java site has a whole
tutorial
on reflection.
So, first of all, let's look at the few different ways you can instantiate and
use ColdFusion Java Objects, we'll use an
ArrayList
as an example, so that we know what we need to support in our own
JavaProxy.
I will use 'createObject' here, but it could also just as easily be
JavaLoader.create(className) to create an instance of the JavaProxy.
array = createObject("java", "java.util.ArrayList").init();
array.add(obj);
- This would be the most common, and generally the 'best' way of
instantiating a Java Object in CF, as it calls the constructor straight
away, with the appropriate arguments, and is the closest you will get, in
style, to implementing a real constructor.
array = createObject("java", "java.util.ArrayList");
array.init();
array.add(obj);
- I've seen this sort of code before... to me, it seems a bit weird to split
out the constructor, but it works, and you always know that the object has
been instantiated.
array = createObject("java", "java.util.ArrayList");
array.add(obj);
- This is what I feel is the 'worst' way of using Java objects in CF as the
no argument default constructor is called implicitly, which means you really
have no control, and it can lead to all sorts of weirdness in your
application if you don't track what has been actually instantiated and what
hasn't.
Collections = createObject("java", "java.util.Collections");
sortedArray = Collections.sort(array);
- This case shows where static methods are called, in which case, there is
no constructor.
Color = createObject("java", "java.awt.Color");
black = Color.black;
- This is where we want to be able to retrieve a static property.
Okay, so we have our work cut out for us! But all this is very much possible!
So, the first decision to make, is that all of our internal methods on the
JavaProxy.cfc are going to start with an underscore. This is so that any
method that we write, doesn't interfere with the onMissingMethod's we want to be
able to pick up. Also as we need to implement a special 'init' method,
that isn't the constructor for the JavaProxy, but instead is a constructor for
the Java object the JavaProxy represents, so we will have a _init(class)
method that instantiates the JavaProxy.
I'm not going to show all the code here, just the relevant parts, but don't
worry, you will be able to see it in the next version of JavaLoader.
So the _init method will do the following things -
Take a Java
Class
as an argument, and store it in state
Store some helpful Java Objects in some setters.
Set the Static
Fields
of the Class the JavaProxy represents to the this scope
Store all the
Method
objects of the Class in a struct of arrays for easy lookup (more on this
later).
First of all, we'll set the Static Fields to the this scope. It is
actually very straight forward -
<cffunction name="_setStaticFields" hint="loops around all the fields andsets the static one to this scope" access="private" returntype="void"output="false"> <cfscript> var fields = _getClass().getFields(); var counter = 1; var len = ArrayLen(fields); var field = 0;
So what are we doing here? Well, we ask the Class for all of it's Fields, then
we look around them, and then interrogate into whether or not they are static.
Once we know they are static, we retrieve their value, using field.get(), and
set them to the same place in the this
scope as they would have been in the Java Object.
We are able to use 'null' on the field.get() because the values are static, and
are not tied to any actual instance of the Class.
So, what would be nice now, is to be actually be able to instantiate an
object! So let's look at implementing our own 'init' method, so that we
can instantiate the Java Class that the JavaProxy represents.
<cffunction name="init" hint="create an instance of this object"access="public" returntype="any" output="false"> <cfscript> var constructor = 0; var instance = 0;
//make sure we only ever have one instance if(_hasClassInstance()) { return _getClassInstance(); }
_buildArgumentArray() simply takes the argument struct, and turns it into an
actual Java Array of the same objects.
So first off, we check to see if we already have created an instance - because
we only want one, otherwise weird stuff could happen. If we do, just give
back the Java instance we already have.
The next line, is a little bit more complicated, so we'll break it down.
The _getClass().getConstructors() returns an array of all the possible
Constructors
that are available for this given class.
The _resolveMethodByParams() method takes a array of Method/Constructor objects,
the arguments that have been passed through to the given method, in this case
'init', and find the best match that it can, and returns it. We'll go into
the details of that in a minute.
Once we have the right Constructor object, to get an instance of the Object that our JavaProxy represents, we call
'newInstance' on it, and pass in an array of the objects that make up the
arguments that the Constructor needs.
And Preso! We have an instance of our Class! We set it to the state of
the Class Instance, and return the newly created instance back out.
Wait! What? Return the new created instance? Why aren't we returning this, that doesn't make sense? Well actually, if you think about it, it does.
We
really would prefer it if ColdFusion did all the heavy lifting when it
comes to the bridge between Java and ColdFusion, not only is it more
performant, but it also provides a greater deal of consistency across
the code base.
So we have code that is:
obj = JavaProxy.init();
obj
is actually an instance of the ColdFusion JavaProxy, and then there is
a much more seamless line between the new CFC JavaProxy, and the use of
the ColdFusion one, which is a very good thing.
That being said,
we need to provide support for all the different types of ways that
Java Objects can be used and created, so we have to also cater for the
other aspects as well.
So without further ado, let's actually fire off some methods! This is where onMissingMethod really comes into it's power! <cffunction name="onMissingMethod" access="public" returntype="any"output="false" hint="wires the coldfusion invocation to the JavaObject"> <cfargument name="missingMethodName" type="string" required="true" /> <cfargument name="missingMethodArguments" type="struct" required="true" /> <cfscript> var method = _findMethod(arguments.missingMethodName, arguments.missingMethodArguments);
if(_getModifier().isStatic(method.getModifiers())) { return method.invoke(JavaCast("null", 0), _buildArgumentArray(arguments.missingMethodArguments)); } else { if(NOT _hasClassInstance()) { //run the default constructor, just like in normal CF, if there is no instance init(); }
Okay, so this is the code that actually takes the methods that are
called on the JavaProxy, and passes them to the Java instance or Class
as appropriate.
The _findMethod() method, returns the Method
that best matches the name of the method that was called, and the
arguments that it has. I will go into detail on that in just a second.
Once we have the correct Method, if it is static, we can then invoke it against 'null', and return it's value.
If
it isn't static, then we check to see if we have a instance of the Java
Class yet, if not, we create one using the default Constructor, which
is the same way that ColdFusion does it.
From here, we are able to invoke the method against the class instance, and return any results that we may get.
Now we can look at the logic that allows us to work out which method matches what in ColdFusion.
Our first step, is to look at the _findMethod method, which is actually pretty simple: <cffunction name="_findMethod" hint="finds the method thatclosest matches the signature" access="public" returntype="any"output="false"> <cfargument name="methodName" hint="the name of the method" type="string" required="Yes"> <cfargument name="methodArgs" hint="the arguments to look for" type="struct" required="Yes"> <cfscript> var decision = 0;
//if there is only one option, try it, it's only going to throw a runtime exception if it doesn't work. if(ArrayLen(decision) == 1) { return decision[1]; } else { return _resolveMethodByParams(arguments.methodName, decision, arguments.methodArgs); } }
throw("JavaProxy.MethodNotFoundException", "Could not find thedesignated method", "Could not find the method '#arguments.methodName#'in the class #_getClass().getName()#"); </cfscript> </cffunction>
The first thing to know is, that _getMethodCollection() returns a
struct of arrays that was set up in our _init(), the key of which is
the name of the methods found in the class. The arrays contained in
the struct have all the Methods that have that name, as there may be
more than one.
So, the first thing we do, is check to see if the
name of the method we need is in the collection of methods we have, if
it is we go and grab the array of methods this invocation could
possibly be.
You will notice that I have written code that
states 'if you only have one option for the method, just return that'.
You may be wondering why, as the parameters of that method may not
match what has been passed in. Well, if that is the case, we will get
a runtime error, which is the same as what we would get otherwise, so
there is not a huge difference here to just say 'let's give this a
shot, if it doesn't work, no big deal', and we save the performance hit
of comparing parameters.
If there are more than one option
available, then we have to start comparing parameters, and this is
where the _resolveMethodByParams() method that we saw earlier does it's
hard work. <cffunction name="_resolveMethodByParams"hint="resolves the method to use by the parameters provided"access="private" returntype="any" output="false"> <cfargument name="methodName" hint="the name of the method" type="string" required="Yes"> <cfargument name="decision" hint="the array of methods to decide from" type="array" required="Yes"> <cfargument name="methodArgs" hint="the arguments to look for" type="struct" required="Yes"> <cfscript> var decisionLen = ArrayLen(arguments.decision); var method = 0; var counter = 1; var argLen = ArrayLen(arguments.methodArgs); var paremeters = 0; var paramLen = 0; var pCounter = 0; var param = 0; var class = 0; var found = true;
if(param.isAssignableFrom(class)) { found = true; } else if(param.isPrimitive()) //if it's a primitive, it can be mapped to object primtive classes { if(param.getName() eq "boolean" AND class.getName() eq "java.lang.Boolean") { found = true; } else if(param.getName() eq "int" AND class.getName() eq "java.lang.Integer") { found = true; } ... else { throw("Ack", "Cannot match this primitive type", "'#param.getName()#' is just not matching"); } } else { found = false; } }
if(found) { return method; } } }
throw("JavaProxy.MethodNotFoundException", "Could not find thedesignated method", "Could not find the method '#arguments.methodName#'in the class #_getClass().getName()#"); </cfscript> </cffunction>
Woah! That's a lot of crazy code... well, it's not too bad once you break it down.
What we are doing is looping around all the possible methods we have available in the decision array, and trying to see if they match the parameters that we have in our argument struct.
First test says, 'if the number of parameters is different, well, we can't invoke this method', and simply passes it by.
From
there, we need to loop around each of the parameters, and see if it can
work with the class that the corresponding argument that matches it's
place.
You're probably looking at the line that reads '_getClassMethod().invoke(arguments.methodArgs[pCounter], JavaCast("null", 0));' and thinking... what on earth does that do? Well, it allows us to get to the Class object of that argument.
This
is very similar to doing a obj.getClass(), however, I can't do that in
all instances. If an argument is a CFC, then it will try and resolve
the method 'getClasss' against the CFC, and most likely throw me an
error. So I have to use reflection!
the _getClassMethod()
(this was setup in our _init()) is the actually Method class that
represents the 'getClass()' method aforementioned, what I then do is
invoke it on the required argument, and this gives me back the Class
Object I need for comparison.
Now, a Class object has
a great method called 'isAssignableFrom', basically, this says 'If the
object is a the same as this Class, or a subclass, or implements this
interface, return true'. This means that if the argument in the method
that have been invoked on the CF side isAssignable to the parameter
that is required for this method, then this method can be invoked
successfully.
The other thing we need to do is map primitive
parameters, such as int, char, boolean, etc back to their Object
representations. The nice thing is that Java will map this back and
forth at runtime for us, so as long as the values match up, we're good
to go.
Once the parameters have all been resolved, we can return the method we have found that works, otherwise, we throw an exception.
That
pretty much covers the JavaProxy.CFC. As I said before, you will be
able to see the code in action once I release the new version of
JavaLoader, but feel free to ask any questions you may have, I will be
happy to answer them.
One thing that I always gets complaints about with JavaLoader
is that you can't delete or rename a .jar file once it is use by
ColdFusion, you either have to restart CF, or name the .jar file
something funky like 'myOwnJar-10042007.jar', so it has a unique name. Now with this version of JavaLoader, that issue is no more!
While the .jar file that JavaLoader uses itself does get locked down (I know.. I can't do anything about that one), but .jar files that you use with JavaLoader no longer are locked by the system!
To note, the JavaLoader cfc has had one of its arguments removed - 'loadedClassPathBias',
simply because it was more difficult to implement with the new code
base, and there didn't seem to be a case for it. By default,JavaLoader
will load the specified jar files before it loads the parent's, if it
has one. Honestly, I don't think this is going to affect a single
person, because I would be shocked if anyone actually ever used the
option.
Please note there are some issues with memory and JavaLoader, so make sure you put your JavaLoaders in the Server scope. See here for more details.
If you find any issues, please post to the forums, or contact me directly.
This is a bug in ColdFusion that can cause memory leaks when using a
java.net.URLClassLoader
to load external jar files. Thus, this can cause memory leaks in
JavaLoader,
Transfer
and any other system that uses this technology. However there is a
workaround for the issue.
To explain the problem, first we need to look at some key issues with a
URLClassLoader. URLClassLoaders are notorious for causing memory leaks,
because, for them to be garbage collected, all instances to themselves
and the classes that they have created
need to be garbage collectible.
This means that if you access a class from a URLClassLoader and hold it
somewhere in memory, then the URLClassLoader can
never be garbage collected.
This is exactly what happens with ColdFusion.
When ColdFusion does some introspection and resolution of ColdFusion code
against a Java object, somewhere, deep inside its hidden internals, it keeps a
strong reference to the Class object that refers to that Java object. This
means that when the JVM comes along to garbage collect the instance of the
URLClassLoader, it can't do it, because ColdFusion has a reference to a class
that it loaded somewhere inside.
So, the memory leak only ever actually happens when an instance of a
URLClassLoader is no longer available to ColdFusion, as it is never then garbage
collected by the JVM.
How does this translate to using JavaLoader? Well, a perfect example of this is
where you put an instance of JavaLoader in the application scope, because
generally it is used as a singleton. JavaLoader (and anything that
subsequently uses JavaLoader) has an instance of a URLClassLoader inside
it. However, when the application scope times out, the JavaLoader CFC may
well be garbage collected, but the URLClassLoader isn't, which can cause a
memory leak.
To note however, in production systems the leak is minimised in situations like
this, as it is often very rare that the application scope will ever time out.
So what is the workaround for this issue? To note, I have been pushing at Adobe
to get a hotfix out for CFMX to resolve this issue, but we can definitely still
use this technique now, without having to worry about memory leaks.
Essentially, the memory leak only happens when the URLClassLoader is no longer
available to CF, i.e. an application scope times out, or something similar - so
we just need to make sure that it never, ever, times out. How can we do
that? why, put it in the Server scope of
course!
Since variables in the Server scope never time out, we don't need to worry about
the URLClassLoader (or JavaLoader) being lost and then recreated, as it always
exists. As long as you put it in the Server scope under a key no one will
ever need to utilise (I like a hard coded UUID myself)! Hence we beat the
memory leak monster!
I have just committed a fix for Transfer that automatically puts the JavaLoader
it uses into the Server scope, so even if your TransferFactory times out, the
JavaLoader never will, which means there is no leak, and the RC2 for 0.6.3 will
have this as well.
Hopefully Adobe will put out a hotfix for this issue, but until then, put your
JavaLoaders in the Server scope.
This is an issue I came across yesterday while doing some work on Transfer that affects JavaLoader , and generally anything that uses an URLClassLoader to load any external class/jar files.
The issue lies with catching Java exception with <cfcatch>. The documentation states, that to catch a thrown Java exception you can specify the type in the 'type' attribute of the <cfcatch> tag. So if we wanted to catch a java.io.IOException, we would do so this way:
<cftry> <!--- do some stuff that may throw an exception ---> <cfcatch type="java.io.IOException"> <!--- do something with it ---> </cfcatch> </cftry>
Now the issue comes when you are using JavaLoader (or anything that utilises any sort of URLClassLoader) and the loaded Java code throws an exception that hasn't been loaded with ColdFusion at start up -
<cfcatch> can't catch the Java Exception by it's specific type!
To show you what I mean, I created a simple test bed.
I create a new Java Exception object, 'com.compoundtheory.test.myException', and a class called 'MyThrower' who's sole job was to throw this exception.
Just to note 'jl' is the JavaLoader instance that has loaded my classes.
My test bed looked like this: <cftry> <cfscript> myThrower = jl.create("com.compoundtheory.test.MyThrower").init();
The result that came from this was that 'java.lang.Exception' was output onto the screen. This means that the ColdFusion <cfcatch> statement bypassed the 'com.compoundtheory.test.myException' type (which is its real type) and caught it at the 'java.lang.Exception' level, which is what 'myException' extends.
This led me to believe that the ColdFusion <cfcatch> can only catch exceptions that are loaded in its class path on start up. To this effect I had two more test beds.
The first test was to get my JavaLoader loaded class to throw a 'java.io.IOException' - an exception that is native to Java 1.4, which comes with ColdFusion, and is loaded when ColdFusion starts.
In this case, 'full' was output, that meaning that the <cfcatch> was able to the exception with the type 'java.lang.IOException', just as it was meant to do.
The final test was when I moved my created classes into the ColdFusion class path, restarted the server, and ran my tests again.
In this case, the output was 'full', that meaning that the <cfcatch> could catch the custom exception, as it had been loaded with the ColdFusion library at start up.
So what does this all mean? Does it mean you can't use libraries with custom exceptions in them? No it does not. But it does mean you need to handle them in a slightly different way when using JavaLoader.
The solution to this problem is to write code that looks like this:
<cfcatch type="java.lang.Exception"> <cfswitch expression="#cfcatch.Type#"> <cfcase value="com.compoundtheory.test.myException"> <!--- do something with it ---> </cfcase> <cfdefaultcase> <cfrethrow> </cfdefaultcase> </cfswitch> </cfcatch>
The nice thing is that the 'type' attribute on the cfcatch variable will be set properly to the type of the Java Exception, so we can use it as a sort of 'filter' to weed out the Exceptions we wish to handle, and rethrow the ones that we don't. It's not the prettiest thing in the world, but it works.
I'm not sure if I classify this as a bug in ColdFusion, but it definitely is an issue if you're not sure how to resolve it.
I realised something today, I've been telling something about the JavaLoader that isn't true.
By default, JavaLoader loads the ColdFusion class path as its parent when loading in custom Java libraries.
This means that if JavaLoader can't find a class that you are loading through it via the Jars that you have loaded, it will look to what ColdFusion has loaded to see if it can find it there.
I had always assumed that if you loaded something, say for example, the newer version of log4j with JavaLoader, for the Jars loaded with JavaLoader, they would see the newer version, rather than the older version of log4j with ColdFusion.
This is actually, completely and utterly false. I apologise for all the frustration I probably incurred on people for making this statement.
Reading the Java docs it explicitly says : 'When requested to find a class or resource, a ClassLoader instance will delegate the search for the class or resource to its parent class loader before attempting to find the class or resource itself.'
So this means that when looking for log4j - the one in ColdFusion actually takes precedence!
This makes sense in the Java world, as it allows for less confusion from the Java engine to be able to resolve classes - but from a ColdFusion perspective it makes life very difficult as we don't have the ability to modify what is loaded at application startup as we would if we had written a Java application from scratch.
A workaround for this is to pass Java null into the JavaLoader as a second variable like so:
This will create a JavaLoader that is not set with ColdFusion as the parent.
I am currently working on JavaLoader 0.3, that will make the precedence on the loaded classes that will solve this problem all up, as well as some parameters to avoid loading the ColdFusion classpath altogether, but I thought I would give people a heads up now to avoid frustration down the road.
One of the annoying things in CF is that when you query a database, all NULL values in the db actually come back as empty strings. This means that if you actually store an empty string in the database, you have no way of knowing if it was an empty string, or a NULL value.
I hit this the other day when working on a database column where both an empty string and NULL values where valid data values, and had very different meanings.
That's when the light turned on, and I realized 'duh! CF is Java, so if the DB value is NULL, the Java call will return NULL, which we can check for'.
So, knowing that (a) a cfquery resultset is actually a java.sql.ResultSet, and (b) that a cfoutput / cfloop call only iterates through the result set, I can can run .getString(columnName) on it, and if the value it returns is NULL, it will return a Java null value.
So what I ended up with was the following isNull() function, that works a treat, and can tell when a real NULL value is returned, not just an empty string:
<cfquery datasource="db" name="query">
select * from table </cfquery>
Huge changes! Massive Changes! The whole thing has been rewritten!
Okay, I'm lying. The only thing I changed is that if you pass in a path to a JAR or a directory when you init() the JavaLoader, it will throw an Exception if the path doesn't exist.
This is simply a useful addition for debugging purposes, especially if you are wondering why you can't load a particular class, or you are getting the wrong version of a class, if you are trying to overload a class that is loaded into ColdFusion.
Oh yeah, I added a 'getVersion()' method too.
If you have any questions, comments or bugs, please send me an email or respond to this blog post.
This is a CF translation of the Java embedding tutorial at http://www.jython.org/docs/embedding.html <cfscript> //set he path paths = ArrayNew(1); paths[1] = expandPath("jython.jar");
//create the loader loader = createObject("component", "JavaLoader").init(paths);
//create the Python Iterpreter interp = loader.create("org.python.util.PythonInterpreter").init(); //set standard output and errors out to the screen interp.setOut(getPageContext().getResponse().getOutputStream()); interp.setErr(getPageContext().getResponse().getOutputStream()); //top line writeoutput("Hello, brave new world <br/>");
//do some python interp.exec("import sys"); interp.exec("print sys"); pyInt = loader.create("org.python.core.PyInteger").init(42); interp.set("a", pyInt); //this now prints out to the screen! interp.exec("print a"); interp.exec("x = 2+2"); //get some values back out of python x = interp.get("x"); writeoutput("x: "& x &"<br/>"); writeoutput("Goodbye, cruel world <br/>"); </cfscript>
Okay, so I'm just hyped up about using the JavaLoader to do some weird and neat stuff - but hey, it's really cool, allright!
I've been doing a ridiculous amount of work recently writing up some Java code that I am looking to deploy with Transfer, and in the process have had all sorts of fun running around exploring the internals of ColdFusion.
The issue however was being able to deploy the code with my library.
(As per usual with these posts, you must note that some of these methods are undocumented and liable to change, yadda ya)
I had my JAR file that I wanted to be able to use with my ColdFusion code, but there were several problems
1. The Java code referenced some ColdFusion, and other related JAR files that were loaded into CF at startup
2. Some of my objects I wanted to be able to create via CF have constructors that require arguments, which is tricky business.
3. ColdFusion likes to keep a 'hold' on any Jar it loads up via URLClassLoader, so being able to redeploy/upgrade the code is tricky, as you can't delete the file without resetting the server.
Okay, so we'll start at the beginning - obviously we have the initial aspect of using a URLClassLoader to load in the external series of .class files, or .jar files, as Spike wrote about back in 2004 . (I'm not going to cover that aspect of it here, there is plenty of examples out now on how to use URLClassLoader out on the blogsphere)
This is all fine and dandy, I load in my class, and I try and create an instance, but it will throw an error - you know why? Because the code in my JAR references Java classes that the URLClassLoader doesn't know about. Namely ColdFusion classes! So it throws a big fat hairy error as it can't see them.
Now, the neat thing is, a URLClassLoader's constructor can take a java.lang.ClassLoader as it's parent - which means it can look to it to try and resolve classes. So off we run to get the ClassLoader for the ColdFusion context.
There are about half a dozen different ways of doing this, but I did it this way -
cfClassLoader = getClass().getClassLoader();
getClass() gets the java.lang.Class object of the current page or cfc, which is very handy for us, as we can grab the ClassLoader for the CF context, and thereby resolve classes that are loaded by ColdFusion when it starts up.
So now, when we create our URLClassLoader, we can create it like this:
classLoader = createObject("java", "java.net.URLClassLoader").init(URLs, getClass().getClassLoader());
Where URLs is the array of java.net.URL that we need to tell the URLClassLoader what directories and jar files to load in.
Okay, so that's all well and good, but problem number 2 arises. URLClassLoader allows us to make calls like:
string = urlclassLoader.loadClass("java.lang.String).getInstance();
To create an instance of a Java object. Which is all fine and dandy if there is a default constructor, BUT so often Java objects like to take arguments when they are instantiated.
I beat my head against this one for a while, as it is a real pain to try and do this through reflection - and then it dawned on me - ColdFusion has already done the hard work for us, why shouldn't we just take advantage?
So I did some digging around and worked out that when you write:
System = createObject("java", "java.lang.System");
The object 'System' is an instance of the coldfusion java object 'coldfusion.runtime.java.JavaProxy'. The JavaProxy takes a java.lang.Class as an argument in it's constructor, and then handles all the aspects of interpreting CF calls on Java objects.
So now, when we load up a class we want an instance of in ColdFusion, we do this -
class = getClassLoader().loadClass("myPersonalJavaObject");
personalObject = createObject("java", "coldfusion.runtime.java.JavaProxy").init(class);
This gives us access to the object as we would normally through a createObject("java", "className"); call.
So now, as we would normally, if we want to create an instance of the object we can call -
instance = personalObject.init();
Just like we would normally. But if we have arguments
instance = personalObject.init(myarg1, myarg2);
And ColdFusion will handle the aspect of translating the CF to the Java object for you.
Finally, the issue that, when you've made your URLClassLoader to a JAR file, the system often won't let you delete the file, as it still marks it as being 'under use'. The way I solved that problem was by using ANT to add a datestamp to the name of my jar flie when it creates it.
This does 2 things:
1. Gives the file a unique name, so I don't have to worry about deleting the previous JAR, and can do so when the server gets reset, or the garbage collector realises that nobody is using the JAR anymore, and releases it.
2. Lets me know when looking at the dir where I store my JAR files which file was created first.
Since I know which one was created first - as long as I load the most recent JAR file to the URLClassLoader first, it will be the first one queried for the class files that are being requested, and therefore you can use the objects that you want.
It also means you could also write some nifty code to ignore the previous version by tokenizing the name of the file, but I will leave that aspect up to you.
This code has yet to be committed to the Transfer CVS repository, but assuming it all goes ahead as planned, look in the package transfer.com.util for a JavaClassLoader to see these techniques in action.
I hope that all made sense, and makes life easier for you while you write your own Java code to run alongside ColdFusion.
Real short post - but something I discovered doing some work on Transfer.
CF Arrays are java.util.Vectors, which means that you can call myNewArray.get(12) on them to get the object at that index.
I kept getting an error saying 'Array 1 index out of bounds', and I couldn't work out why, when I knew I only had one object in the collection.
The thing I totally forgot was, Java arrays are indexed from 0, and ColdFusion arrays are indexed from 1.
So when doing a 'get()' on a Array, you have to remember to refer to your Array as a 0 indexed array, not a normal one like usual... that is if you are brave enough to mess around with Java trickery ;o)
Scaling a BufferedImage produces a Image, which can't be encoded into JPEG, so you need to create a new BufferedImage with the given Image.
So, here is the code I used as a test case to take an image of a Nissan 350z and scale it down by a quarter:
<cfscript> //need this to do all the JPEG work JpegCodec = createObject("java", "com.sun.image.codec.jpeg.JPEGCodec"); //reading in file inputStream = createObject("java", "java.io.FileInputStream").init(expandPath("nissan.jpg")); //pushing out the file outputStream = createObject("java", "java.io.FileOutputStream").init(expandPath("nissan1.jpg")); //decodes the JPEG decoder = jpegCodec.createJPEGDecoder(inputStream); //give us an image to scale image = decoder.decodeAsBufferedImage(); //make it 1/4 the size height = round(image.getHeight() /4); width = round(image.getWidth() / 4); //this is java.awt.image.Image, but we need a java.awt.image.BufferedImage! scaledImage = image.getScaledInstance(JavaCast("int", width), JavaCast("int", height), image.SCALE_DEFAULT); //create a new BufferedImage newBufferedImage = createObject("java", "java.awt.image.BufferedImage").init(JavaCast("int", width), JavaCast("int", height), image.TYPE_INT_RGB); //draw the image on the buffered image graphics = newBufferedImage.createGraphics(); graphics.drawImage(scaledImage, 0, 0, Javacast("null", "")); //encode it encoder = jpegCodec.createJPEGEncoder(outputStream); encoder.encode(newBufferedImage); //close off the resources outputStream.close(); </cfscript>
So there you have it - JPEG scaling without using an external Java library. Enjoy.
I was requiring a synchronized First in First Out (FIFO) Queue, to store some CFCs for reuse, and I was doing some digging around to see if there was anything already implemented, or if my best bet was to go for just using a synchronized linked list for the Java Collections Framework.
I already knew that ColdFusion had quite a few org.apache libs sitting underneath it (xalan, log4j, axis etc), so I figured I would do some checking and see if the Collections where also there, and strangely enough, they were!
There is a whole lot of very useful collections in here, including Bags, Buffers, MultiMaps, PriorityQueues... all sorts of stuff.
So if you are looking for a collection that you can't seem to find in the regular ol' Java, I suggest checking out the Commons Collection - you should be able to find what you are looking for.
So I'm parsing through some response text I get from a CFHTTP I'm firing off, and I figure - the easiest way to parse through this thing, is actually going through it line by line.
There are a variety of ways I could have done this particular endevour, but I figured I build myself up a java.io.BufferedReader that I would pass my String into (via a java.io.StringReader), and I could loop around the thing till my heart was content.
Only issue is this - the only way to tell if the BufferedReader is done, is that it returns null from it's readLine() method.
Doh. CF doesn't handle null values all that well.
So I figure - maybe it will return an empty string? Who knows. I'll set it up like I would in Java and see what will happen.
Which unfortunatley doesn't work too well - as CF won't handle the evaluation inside parenthesis, and thinks the = is my mistake for 'eq'. So we switch over to a do ... while loop instead, and get:
do
{ line = bufferedReader.readLine();
} while(Len(line));
Which works - except it returns back an error: 'line is not defined'.
Which I realise (but fail to understand) is that when Java returned the null value out of the method, it actually removes the line variable from the scope of the method! Woah!
So in the end, this is what I end up with, and it works a treat:
I'm doing some CF R&D (basically I have a little free time, and I was thinking about playing with some things) - and I needed a hash code that would be specific to an object- more often than not a complex object like a Struct or Array.
After a bit of experimentation I found that most of the complex structures that are utilsed in CF (Vectors and Hashtables) return either 0 or 1 as a hashCode(), which is not all that useful.
I finally came along to java.lang.System.identityHashCode() which gives you a specific number that is unique to that object. It is important to note that if the object changes (i.e. you append a value to the array for example) the value of identityHashCode stays the same.
So if you were to compare two identiyHashCodes from different objects, if they were the same, you know them to be exactly the same object.
So of course, i came up with a little UDF for this:
EDIT :::: made a small mistake when I first wrote this UDF as I was walking out the door... fixing it now.
function identityHashCode(object)
{
var system = createObject("java", "java.lang.System");
return system.identityHashCode(arguments.object);
}
Nothing very exciting in the UDF, but an interesting piece of code never the less, if you are trying to muck around with some of the inner workings of Objects in CF.
I just wanted to list some of my favourite Java shortcuts that I like to use when using ColdFusion
java.lang.String.length() - Yes I could do this with Len() from cf, but typing myString.length()seems to be so much easier and faster for me.
java.lang.String.getBytes() - a useful way of working out how many bytes in a string. This can be interesting to use to check approximatley how much memory you are using when storing long strings in a shared scope.
java.util.Date.before() and .after() - Yes, I can use dateCompare, but I simply find if(myDateOne.after(myDateTwo)) {..} so much easier to read (and all CF date objects are Java date objects).
java.util.Vector.addAll() - Since all CF arrays are vector, this is a easy way of adding an entire array into another without having to manually enter every single item. i.e. <cfscript>
a = arrayNew(1);
a[1] = "1";
a[2] = "2";
a[3] = "3";
b = ArrayNew(1);
b.addAll(a); </cfscript>
I think that's all the ones I usually use at the moment - what other Java snippets do you use to save yourself time?
I found these two really nice articles on Business Objects on Java Boutique yesterday, and thought I would share them, considering my previous post on using OO with ColdFusion.
Business Objects tend to find their way into almost every type of application development that utilises an OO approach, so understanding the concept (and you may already be doing it without knowing it) is a pretty good idea.
The second article goes through a example application, and goes through alot of the considerations to be made when utilising business objects in an application.
Obviously the articles are Java related, but due to the general nature of Business Objects, as well as the myriad of frameworks that support BO's under Java, it tends to stick to pretty vendor non-specific details, so is a good read for ColdFusion developers as well.
This is a bug that I've been getting on here for a while, and I've been meaning to fix it. Strangely enough today is the day!
The issue comes when either an older browser, or a badly implemented RSS reader picks up my links.
Since both my webpage is and my RSS feed is XML (XHTML), I will turn a link that looks like /?action=displayPost&ID=99 into one that looks like /?action=displayPost&ID=25, as '&' is not a valid XML element.
This is normally no issue, but sometimes I do get requests that still retain the & within the URL, as it hasn't been translated back to the normal ampersand.
My original thought was to fix the issue with a <cflocation>, however I didn't feel that would be seamless enough, and if there was form data going through, that wouldn't work either.
So I used getPageContext().forward() to forward on the request, after I fixed it by replacing all '&' with '&' in the url query string, and because the forward() method simply pushes the java HttpServletRequest and HttpServletResponse objects through to the new request, everything is totally seamless. Form data gets passed through, and the user won't even see a URL change in their browser.
Here is the code below - I have it in a custom tag called 'fixAmp.cfm' that gets called in my Application.cfm
<cfscript>
//just for cleanup
StructDelete(request, "fixAmp"); </cfscript>
</cfif>
The reason I have the request.fixAmp flag in there is because on some servers, if you forward the request, the CGI data will stay the same. Hence the tag will go into a infinite loop as it never think's it's fixed the ampersand.
And presto, you never have to worry about issues with useing & in your links again!
This was one question I was hitting myself for not asking after the recent MXDU in Sydney.
Point: ColdFusion sits on top of a J2EE engine
Point: This is very neat, because it means we have the power of Java at our hands to do low level work we would not otherwise be able to do in CF.
Point: Any time you want to do Java code that integrates with already existing ColdFusion code, there is a whole lot of guesswork and introspection into undocumented features of CF so that we can leverage it's power in ways that we need. (i.e like my post on attempting to add a HttpSessionBindingListener to a CF Session).
Question: Why doesn't CF ship with Javadoc API documentation of the Java classes it is built on? It would make all of our lives a helleva lot easier.
It's not like I'm asking for CF to become open source (which I don't even want), but simply make available the information we can already get through Java Reflection in an easy to use format.
I recently unzipped the cfusion.war archive from the J2EE deployment and made it so I could compile a copy in my Jave IDE (which was a tad trickier than it seemed). I did this so I could compile some servlets against the already existing codebase, and make sure there were no conflicts. Admitedly I also did it so that I could grasp a much finer notion of the underpinnings of what ColdFusion really is, and get to put my hands in as many of its internals as I could. For those of you who do Java/J2EE work as well, I would say try this out, it gives you a whole slew of new ideas in terms of extending ColdFusion.
There is a small part of me (and for the record, I am in no way advocating this) that is tempted to take a decompiler to the codebase of CF, just to see what makes it tick. Just to be clear on this point - I am quite aware this would be quite illegal, and I have not done so. That being said, the only reason I would do so, is simply so that I can understand CF at a much lower level, and simply become a better ColdFusion Programmer. The purpose of this discussion is not for whether or not decompilation is ethically/legally good or bad.
So to end on a positive note - while this does sound like a gripe at heart, really I'm just asking for something that in some ways already exists, and I think would be a very large resource for all the developers who work with ColdFusion today. It would mean we would all stop grasping at things we think are there in the code base, and can concentrate on really building on what we know is there.
So in case any MM guys happen to read this - Can we have it?
Here I was thinking I was really clever - but I guess I'm not as clever as I had thought.
I wanted to create a way for CF'ers to have a way to determine when a Session had ended - I could do it in J2EE, so I could do it in CF? Right? CF just sits on top of Java, so it should be easy.
So I created a Java class that implemented the HttpSessionBindingListener interface and set it up to call a series of URLs when a session was either timed out, or the user logged off.
I turned on 'Use J2EE Session Variables' within the cfadmin app and wrote the below CF test code:
Using this and subsequent test code, I found that neither valuBound() or valueUnBound() was called on the my Java object.
This led me to believe that CF only implements a mechanism for sharing data between J2EE sessions and itself, and does not actually utilise the actual J2EE session objects themselves.
Apon further examination (thanks to getJavaMetaData), I discovered that the CF session object doesn't even come close to the javax.servlet.http.HttpSession interface, so there was no way this was going to work in the first place. (It is in face a coldusion.runtime.j2eeSessionScope object).
It's a pity that this hasn't worked, it would have been very handy - and I know how long CF'ers have wanted the capability to know when sessions had ended.
So I guess in conclusion - we now all know, that when it says in the CF admin 'Use J2EE session variables' - it's really not telling the entire truth.
Naughty Naughty. ;o)
Just for fun these guys over at IBM talk about some ways of sharing J2EE and CF session information. An interesting read never the less.
Had an interesting problem float past me the other day, that had me thinking a little outside the CF square.
Situation: We have an existing HTML website, and we are moving it to a CF website.
Problem: We don't want to break any of our existing HTML links. I.e. if you we have http://www.mysite.com/page.html it should stay the same - so no sneaky .cfm extensions.
Interesting Point: It's a standalone CF Instance running on a J2EE server. Hmnnn...
So the obvious starting point is to map the CF Servlet to *.html - much like you could already do within a IIS configuration, nothing all that interesting there.
Obviously there is now some new CF magick, that allows for the pulling in of content (basically through a single CF Custom tag, that does some processing dependent on the URL).
But given the standard CF setup, that would mean we have to create a CF page for every old html page. That is seriously gonna suck, and is a lot of hard work, for not much payoff.
So of course, I got a' thinking and said - 'wait a minute, why can't we use a servlet to fake the html page, and then pass the URL information to a single CF page via the request scope?'
I.e. do something like this:
public class CFForward extends HttpServlet
{
private static final String FORWARD = "forward.cfm";
public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
{
//grabs the servlet path
String servletPath = request.getServletPath();
/*
Sets the request scope parameter called 'servletpath' that is accessable from the
Coldfusion page. Key must be LOWERCASE!!!
*/
request.setAttribute("servletpath", servletPath);
//forward on to the coldfusion page.
this.getServletContext().getRequestDispatcher(FORWARD).forward(request, response);
}
}
Then add this Servlet mapping to *.html (and get rid of the CF one) - and presto, seamlessly drives all *.html requests over to "forward.cfm", along with the relevent details in it's request scope.
That AND all the URL and FORM scope data is still there as you would need it.
(Note: my version pulls in details from the web.xml, and does some other stuff but I figured I'd keep it simple)
So now you have what looks like a static HTML page, which is really a servlet running a ColdFusion page behind the scenes.
Yup, I don't believe what a URL tells me anymore. Where have the days gone when .html was static, .cgi was perl, and .exe was crazy ;o) You always knew where you stood with a webpage, just by looking at it's extension. Now it's all smoke and mirrors.
The only problem I've hit so far - on the website, the web stats package picks up 404 errors, however, with a servlet mapped to *.html, ANY page with ends with a .html will get picked up by the servlet. If relevent (i.e. no content for that URL), I need to be able to push a 404 error to the Server (getting it to the client isn't hard, you don't even have to use a real 404). Not sure how I'm going to do that yet.
Okay, maybe hate is a bit of a strong word, but seriously, having done some (serious?) XSL development under Java, the XMLTransform() of Coldfusion does leave a little to be desired.
Inability to pass in xsl:param values externally
This one is my primary gripe. I don't understand why you can't do this natively in ColdFusion. In Java, when you create your XML transformation via XSL you can pass in values that are defined in the top of the XSL document like so:
<xsl:param name="root"/>
Which means I can use that variable later on in my XSL stylesheet - the primary example being being able to pass in the root URL of your application, so you can build links from it, so regardless of where you run this XSL stylesheet, you know images and links will still be relative to the root. (Admitedly there are other ways to do this, but this is just an example).
Within the current power of CF, you would have to build your XSL at runtime, and add in these elements yourself. Personally I see this as kind of cludgy, as I prefer to keep my XSL files in a flat file somewhere.
XSL Files must be read in to a variable before they can be used This means that before you use an XSL file, you have to <CFFILE> it in before you use it. 'No big deal' I hear you say, cache it in a scope somewhere and then reuse it later... yes yes, this is all valid, except for one small thing
<xsl:import href="modularXSL.xsl"/>
If you do this - you can no longer do relative XSL imports, which pretty much destroys any chance of doing modular XSL development. Considering that you can't use a <xsl:param> to dynamically pass through the root path, that does mean you are left with developing your XSL at compile time... (which means it's not a file anymore anyway, and can't be used by other xsl stylesheets), or hardcoding your logical path into the XSL stylesheet itself (yuck!).
But what do we do now? Okay, so I've had my major gripe, and I did have a good winge for a little while about this before I got mad enough to actually look for a solution. There is a solution to one of the issues, there is a xslt() function that can be found at cflib.org.
This runs natively from CF, however does not handle my issue with using native <xsl:import>.
So of course, I didn't get even more mad - I decided to get down and dirty with some Java and came up with my own XSLT() funtion that uses the underlying Java engine (CF uses Xalan it seems under the hood for anyone that cares) that can take either (a) a XML / XSL string, or (b) a file path to a XML / XSL file.
This means it can do BOTH parameters, AND relative xsl importing!
Arguments: xmlSource - either a valid xml document as a string, or a absolute file path to a XML file. xslSource - either a valid XSL document as a string, or a absolute file path to a XSL file. stParameters (optional) - a structure of xsl:param elements to pass through where the key is the name of the param, and the value is the value of the param being passed through. Do note that StructInsert() will need to be used as param names are case sensitive, and otherwise the struct key value will be in uppercase.
Example: This can be run a variety of ways:
<cfxml variable="xml"> <!--- valid xml doc ---> </cfxml>
<cfscript>
var source = ""; var transformer = ""; var aParamKeys = ""; var pKey = "";
var xmlReader = ""; var xslReader = ""; var pLen = 0;
var xmlWriter = ""; var xmlResult = ""; var pCounter = 0;
var tFactory = createObject("java", "javax.xml.transform.TransformerFactory").newInstance();
//if xml use the StringReader - otherwise, just assume it is a file source.
if(Find("<", arguments.xslSource) neq 0)
{
xslReader = createObject("java", "java.io.StringReader").init(arguments.xslSource);
source = createObject("java", "javax.xml.transform.stream.StreamSource").init(xslReader);
}
else
{
source = createObject("java", "javax.xml.transform.stream.StreamSource").init("file:///#arguments.xslSource#");
}
transformer = tFactory.newTransformer(source);
//if xml use the StringReader - otherwise, just assume it is a file source.
if(Find("<", arguments.xmlSource) neq 0)
{
xmlReader = createObject("java", "java.io.StringReader").init(arguments.xmlSource);
source = createObject("java", "javax.xml.transform.stream.StreamSource").init(xmlReader);
}
else
{
source = createObject("java", "javax.xml.transform.stream.StreamSource").init("file:///#arguments.xmlSource#");
}
//use a StringWriter to allow us to grab the String out after.
xmlWriter = createObject("java", "java.io.StringWriter").init();
There you go - copy paste that funtion, and now you have every all funtionality you could probably ever want when doing an XSL transforamtion.
I will probably shoot this off to cflib.org at some point soon, so you can search for it there as well.
I must say that I am loving the Java integration with CF, it has definately enabled me to do many more interesting things with CF I was previously never able to do before.
If you have any questions / comments / bugs, drop me a line, or post a comment.