Archive | ImageMagick RSS for this section

DMP Photo Booth: Underwater

You’ve heard it before: “Premature optimization is the root of all Evil.” Capital Evil. So you go on about your day, arranging the ones and zeros in pretty christmas tree shapes and suddenly the day arrives: your program is slow as molasses. What are you going to do now?

Last monday was that day for me, and I’ve been underwater ever since. “Why is this happening to me?!” I thought. While not prematurely optimizing, I thought I did things right. I have no nested for loops. I’m not using an array when I need a list. Threads aren’t modifying the UI willy-nilly. Why has God forsaken me?

The Symptoms

I first noticed it while working on the printer module. After the program is open for some length of time, my whole computer begins to lag. Not just a little bit either; things completely fall apart. In the space of about 5 minutes, the computer becomes unusably slow. Killing the Photo Booth process doesn’t help; only physically shutting the computer off helps. Of course, the computer is so slow that I can’t use the shutdown option; I have to press The Button.

At this point, I feel some context is in order. I had been trying to figure out how to make my printer print on photo paper. Apparently printing is one of the areas Linux still hasn’t caught up to windows on, so this was proving to be difficult. After printing a few strips, I realized that my low-res photo strips weren’t going to cut it, so I bumped the resolution from 100 pixels wide to 1000. It was then that I noticed things were off.

Ten years of troubleshooting experience kicked in: “what changed?” I thought. The obvious answer was the image size. Clearly my photo strip assembly algorithm was operating at O(n^n^n) or something. What can be done?

Doing It Wrong

I took a look at my assemble strips function. After poking around for a while, I zeroed in on something that had been bugging me for a while. I had been using a function MagickResetImagePage combined with MagickCoalesceImages to composite images over each other. I had decided to use these functions before I knew this operation was called “compositing”, and I had found them in a tutorial on making animated .gif files in MagickWand. At the time, I was never really happy with this implementation, so I went back to the API docs to see if there was a function with “composite” in its name. There was.

MagickCompositeImage is a lot more intuitive to use than MagickResetImagePage. It doesn’t have that Magickal formatting string that MagickResetImagePage uses, it just takes coordinates. Perhaps this was the solution to my problem. I refactored, and recompiled.

Still broke.

Measure, Don’t Guess

That old gem: I’m sure you’ve heard it too. I decided that maybe this was my best course of action. I decided it was time to learn how to use this Valgrind thing all the Cool Kids are talking about these days. For those of you not in the know, Valgrind is a utility that will tell you various things about your program. The most important/most well-known thing that it can do for you is identify memory leaks. Thinking that prehaps I have a memory leak, I installed Valgrind and got to work.

It turns out that GTK has more than a few memory leaks. Allegedly this is due to the fact that it doesn’t cleanup on exit, relying on the OS to free the memory on program termination. While the general consensus is that this is fine, it doesn’t help us. The folks at Gnome are aware of this, and there is even a Wiki page on ways to mitigate this. The cliff’s notes version of that page being: “Just search for ‘definitely lost'”.

Armed with this piece of wisdom, I set off. I ran the Photo Booth in Valgrind, and examined the results. Valgrind actually turned up some memory leaks, which I corrected. Maybe now we’re set!

Nope.

Breaking Out The Profiler

This is what they usually want you to do when they tell you to Measure. Unfortunately for me, NetBeans’ built-in profiler is only for Java. After some google searching, I found gprof. Gprof is a pretty bare-bones profiler. It does what it says and not much else, which is fine. I hooked my program into the profiler and got to work. The results? Nothing. My two GTK idle functions ran some 7 million times, returning basically immediately each time as expected. Every other function performed as expected.

What now?

Trying The Process Monitor

Having run through Valgrind and GProf, coming out empty-handed, I was at a loss. I got into development because I wanted to fix my own broken code instead of mitigate somebody else’s, and fix it I will. Luckily I have 10 years of sysadmin experience to fall back on. I dusted off my process monitor and got to work.

I fired up DMP Photo Booth, and watched it in the process monitor. I pushed the button. I pushed it again. And again. memory use rose and fell predictably as the strip was assembled, but CPU usage stayed relatively low. Then boom!

I tried again, this time doing literally nothing. Still my computer sputtered and died. I killed the process, but again it was too late.

But wait, isn’t the OS supposed to clean up after me when my process ends? Something fishy is going on.

Have I Mentioned That Threads Are Hard?

Having eliminated all other possibilities, I was forced to consider that I was having a threading issue. “But I was so careful!” I thought. Shortly thereafter I noticed it: I was getting random pthread mutex errors on my console. Clearly I had a threading issue on my hand. Was I spawning extra threads? Was something not releasing its lock? Was I being victimized by gremlins? I set a break point on line one of main() and fired up my debugger. It was time to see just what was being done when nothing was being done.

So, I stepped through my program. Whenever I got to a g_thread_new call, I made sure the thread function was solid. Finally, I got to my g_idle_add calls. I had two of them, one to monitor the status indicators, and one to retrieve photo strip thumbnails. Both of these functions pop from a result from a GAsyncQueue. These Queues are fed by worker threads. I thought back to my profiler output and remembered how often these are called. Looking a few lines down I saw a call to g_timeout_add_seconds. This function is basically adds an idle function, but is only called at most X seconds. Maybe replacing the g_idle_add calls with g_timeout_add_seconds was my answer. I refactored and reran.

Nope.

Well, crud. “Are these functions even my problem?” I thought. I commented them out, recompiled and reran.

Fixed.

“So, what’s the difference?” I wondered. All three of these functions rely on the same basic behavior: pop from a GAsyncQueue some result placed there by a worker thread. I looked at the three threads: the thread that was working properly calls g_async_queue_ref/unref, and the two that don’t work do not take a reference, instead accessing the static global variable in their module. I refactored all thread functions that access a GAsyncQueue to take a reference and work on their local copy only. I recompiled, reran, and went to bed. 46,100 seconds later, everything was humming along just fine.

Wait, So I Just Had To Increment A Reference Count?

It certainly seemed odd. That’s like your car not starting if the headlights are out. Sure, they’re important, but the car should still start right?

Looking through the source of glib didn’t help. So far as I can tell, all that does is increment the reference count, and return a pointer. I turned to the documentation, which says “… Whenever another thread is creating a new reference of (that is, pointer to) the queue, it has to increase the reference count (using g_async_queue_ref()). Also, before removing this reference, the reference count has to be decreased (using g_async_queue_unref()). …” While not definitive, this certainly seems to indicate that taking a reference is important.

Frankly, I’m not happy about this answer. This is just the sort of magic solution that I hate; it’s fixed, but I’m not sure why. For the time being, I won’t dwell on it. Moving forward, I’ll be sure that my threads take a reference of a GAsyncQueue before calling methods on it. At some point when all of this is said and done, perhaps I’ll investigate this mysterious reference count.

I have taken away from this a new appreciation of just how brittle threads are. Sure, they are powerful, but shooting yourself in the foot with a 50 cal hurts a lot more than with a 9 mm. I’ll have to be more careful.

It was also a good introduction to GProf and Valgrind. Expect blog posts on the usage of each of these tools soon!

DMP Photo Booth: Deep Magick

After working on DMP Photo Booth for a few months, the day came when I needed to implement actual functionality. It’s the day we all dread, but for me this was no longer some looming menace; it was time to stop fiddling around in my framework and actually build on it.

More specifically, it was time to figure out how to turn ~5 images and a background into a photo strip. After checking to make sure GLib and GTK didn’t provide this functionality (GdkPixbuf almost cuts it, but as far as I can tell, it can’t layer images over each other), I turned to google. After some time, I settled on my library: ImageMagick.

ImageMagick bills itself as sort of a command-line PhotoShop. I was suspicious as well, but that’s neither here nor there. The thing about ImageMagick that interested me is its language bindings. ImageMagick provides a library for many languages, including two for C: MagickCore for “Wizard-level developers” and MagickWand for us chumps. Being a chump, I decided to go with MagickWand.

NetBeans configuration

This was relatively straightforward. If you got GTK set up, this should be no problem for you. First, ensure you have the WagickWand development headers. On Ubuntu, this can be accomplished by the following command:

sudo apt-get install libmagickwand5 libmagickwand-dev

On my system, libmagickwand5 was already installed, so I downloaded the headers and got to work.

Next, in NetBeans click Tools->Options, click on the C/C++ tab, and click on Code Assistance. Add the location of the ImageMagick headers (/usr/include/ImageMagick for me). Click OK.

Next, we need to set up our project. Right click your project and click Properties. Click Build->C Compiler. Under Additional Options, add MagickWand to your pkg-config --cflags string. Click Build->Linker and do the same with your pkg-config --libs string.

You are now ready to conjure some magick!

Conjuring Some Magick

Now, let us put on our robes and wizard hats; it’s time to do some magick! Let’s go over a function that will overlay a resized image over a larger background. You can find the actual production photo strip function on Github.

void cast_magick_spell() { MagickWandGenesis(); ...

Before we can do anything, we must initialize MagickWand. “MagickWandGenesis()”, you ask? Of course the function would be called MagickWandGenesis, what kind of silly question is that?

... MagickWand * background_wand = NewMagickWand(); MagickWand * working_wand = NewMagickWand(); MagickWand * final_wand = NULL; ...

The MagickWand * is the main object passed around in a MagickWand application. Working with MagickWand requires some juggling of these pointers, which is why we have 3 of them.

... if (!MagickReadImage(background_wand, "/tmp/background.jpg") { ...

This function reads an image from a file. It returns MagickTrue on success, and MagickFalse on failure. If it returns MagickFalse, we have some clean-up to do…

... ExceptionType exception_error_code; char * exception_message = MagickGetException( background_wand, &exception_error_code); ...

MagickWand makes use of “Exceptions” throughout, so if a function fails, you can most likely pull an exception out of it using MagickGetException. Like any exception, it is up to you what to do with them. Since I’m using GLib, I’ve been wrapping them in a GError and propagating them up. This is actually very easy to do; your error code and message are already there. All you need to do is G_DEFINE_QUARK your error quark and throw it in there. For the purposes of this function, I’m just going to use printf, do some cleanup, and return.

... printf("Oh no! Exception %d: %s\n", exception_error_code, exception_message); MagickRelinquishMemory(exception_message); DestroyMagickWand(background_wand); DestroyMagickWand(working_wand); return; } ...

Nothing particularly shocking here. We printf our message, free the exception_message string using MagickRelinquishMemory, free our MagickWands using DestroyMagickWand, and return.

Assuming we make it past this block, background_wand now contains the background image. Next, we load the foreground image:

... if (!MagickReadImage(working_wand, "/tmp/foreground.jpg") { /* * Exception handling omitted * for brevity. You should still * do it here... */ } if (!MagickResizeImage(working_wand, [WIDTH], [HEIGHT], LanczosFilter, [BLUR])); { /* More error checking... */ } ...

After loading the foreground image, and doing our error checking we attempt to resize the image using MagickResizeImage. This function takes several parameters:

  • MagickWand * working_wand: The MagickWand to operate on
  • size_t [WIDTH]: the width to set the image to
  • size_t [HEIGHT]: the height to set the image to
  • FilterTypes LanczosFilter: the filter to use to resize. There are a list of them in the API documentation. Discussion of these is outside the scope of this post.
  • double [BLUR]: The blur factor to apply. 1.0 is no change. The further 1.0, the blurrier the resulting image.

Like many calls in this library, this function can return MagickTrue or MagickFalse. If it returns MagickFalse, something threw… Next, we adjust the position of the image…

... if (!MagickResetImagePage(working_wand, [RELATIVE_PAGE_SPECIFIER])) { /* * You'd think one of these wizards * could have written a function that * doesn't throw... */ } ...

This function takes some explaining. The second parameter: char * [RELATIVE_PAGE_SPECIFIER] is what’s doing the work here. This is a Magickally formatted string that looks like this: "100x100+15+15!". Let’s examine this as a printf formatting string:

"%dx%d+%d+%d!"

We have 4 integer tokens here. The first token is canvas width, and the second is the canvas height. Note that these will not resize the image, so don’t get any bright ideas about eliminating the previous resize call. The third and fourth tokens are offset from X and Y respectively. These are what we’re really concerned about here. This will allow us to position our foreground image over the background.

Also, don’t forget to check for exceptions!

... MagickSetLastIterator(background_wand); if (!MagickAddImage(background_wand, working_wand)) { /* exceptional! */ } ...

We’re almost there! Now we have to add the images from working_wand to background_wand. These MagickWand objects are lists of images. Like most lists, they have iterators. The call to MagickSetLastIterator sets background_wand’s iterator to the last image in the list. Any images add will be added after this image. next we call MagickAddImage which adds copies of working_wand’s images into background_wand. As before, don’t forget to check for exceptions.

... final_wand=MagickCoalesceImages(background_wand); if (!final_wand) { /* You guessed it! */ } MagickSetLastIterator(final_wand); ...

Now we need to combine all our images into one single image. This is accomplished by calling MagickCoalesceImages which returns a new MagickWand with all of our images combined into 1. The MagickWand used for this call remains unaffected. Obviously, if final_wand == NULL, something threw.

After this is done, we need to set the iterator of final_wand to the Last Iterator, or the next step doesn’t work as advertised…

... if (!MagickWriteImage(final_wand, "/tmp/final.jpg")) { /* wait for it! */ } ...

Shockingly, this function writes your new image to a file. Make sure you check your exceptions.

... DestroyMagickWand(background_wand); DestroyMagickWand(working_wand); DestroyMagickWand(final_wand); MagickWandTerminus(); }

…and were done! Clean up your pointers and call MagickWandTerminus to finalize the MagickWand library. If you browse to /tmp, you should have a newly created final.jpg if all was well!

Would I Do It Again?

Sure.

ImageMagick was a decently easy to work with library. The documentation wasn’t amazing, but it was tolerable. I’m still a little hazy on the use of the [RELATIVE_PAGE_SPECIFIER], but it’s working so far. One nice thing about the docs is that there are many examples. If the docs don’t explain something, you can look up an example and get an idea of how things work.

The only really big issue I have with this library is how it handles exceptions. This is an issue that I’ve touched on before; it is all too easy to forget to check a return code. I went down that road with DMP Photo Booth, and I’ve since rejected it. I spent an entire day refactoring my program to use GError.

MagickWand has exceptions, they’re just really easy to ignore. While writing this blog post, I caught several instances of unchecked return values in my 1 function that uses MagickWand. Tomorrow I plan to fix this, but it’s time I could be spending on something else.

If GError is Java’s checked exceptions with all it’s order and verbosity, then MagickWand’s exceptions are the Wild Wild West of C++’s unchecked exceptions. If you’ve spent any time working with C++, this has almost certainly bit you; some function doesn’t document if it throws, and your program magickally starts crashing because you didn’t catch something. Bad times are had by all. Sure, you could throw a try/catch block in main() and catch all exceptions to keep from crashing, but at that point your program is a dead man walking. Best to put it out of its misery…

Personally, if I ever write a personal project in C++ again, I’m likely to disable exceptions in my program; they’re just more effort than they’re worth. Maybe I’d even use GError in my C++ app if I could convince myself that I’m not a bad person for using a C library in C++.

Regardless, one blemish on an otherwise pleasant experience is no big deal. Here’s to a successful foray into the land of High Adventure!

%d bloggers like this: