Archive | UNIX RSS for this section

What Is The Deal With Open Source?

So maybe you’ve heard talk of something called “Open Source”. Maybe a friend or family member told you about it, or you heard about it on the internet. Maybe you use it. Usually, when somebody professes the virtues of open source, they speak of it as this great force of good protecting the masses from the Evil Corporations. If Superman were a piece of software, he would be open source, here to protect you from Lex Luthor’s new giant mechanized monster: The Micro$oft Abominatron.

The point is that these arguments are emotional. Conventional wisdom would tell you that this is because Computer People are bad at explaining things. The fact is that open source is a difficult concept to internalize. It seems ridiculous; if your application is so great, why put it out there for free for anybody to use or modify? Why not sell it and become the next Bill Gates?

There are good answers for these questions, and in this post I’d like to discuss some of them. This is a post for the lay person, and as such I’ll keep the technical jargon to a minimum. That said, when you speak in analogies, little important details can get lost in translation. Please bear with me. That said, let’s get down to it.

What's The Deal With...

So, what is the deal with Open Source?

A Brief Overview

An open source application is an application whose source code is freely available to modify. These applications are licensed under one of various open source licenses. These licenses are similar to the one you click “accept” to when you install a program. The license describes the restrictions on what you can do with the open source application. There are very permissive licenses, such as the MIT and BSD, where they basically say “do what you want with this software, just give credit where credit is due” (allowing code to be used anywhere).

There are also very restrictive licenses, such as the GPL license which basically says “you may do what you want with this software, so long as any derivative works are distributed under this same license” (effectively ensuring the code is only used in other open source projects).

Whatever license an open source project chooses, the intent is the same. The code for the application is freely available somewhere, and you are free to use the code for the software in your project, so long as you respect the terms of the license. You could take an open source application, change its name, and release it as your own if you wanted to. Just make sure you respect the terms of the license. However, this isn’t typically done for various reasons.

While open source means that the code is free for anybody to access, this doesn’t mean that just anybody can make changes to the original project. Typically the group behind a project will regulate who can make changes, and how changes get made. Some organizations are quite open about accepting changes from the community. Some don’t accept changes at all. In this way, the “owner” of an open source project controls the software, and maintains the quality of the product.

Why Should You Care?

At this point you may be thinking “that sounds all fine and good, but why should I care about any of this?”. It does seem awfully warm and fuzzy, but there are some very good reasons you should care.


The most obvious reason is money. Suppose you want to type a document. You’d need a word processor, the obvious choice is Microsoft Word. So you go to Microsoft’s online store and search for Word. You click on the link for Word and see that it costs $109.99. Just for Word. The entire Microsoft Office bundle costs $139.99. They will also allow you to pay them $99.99 every year for Office.

What if I told you that there was an open source office suite called LibreOffice that you could download right now for $0? LibreOffice can do everything office can do, but costs nothing. I use it for all of my word processing needs, and I can tell you I will never pay for another copy of Microsoft Office ever again.

Don’t want to spend $19.99/month for Adobe PhotoShop? Download GIMP for free. Don’t want to spend $119.99 for Windows? Download Ubuntu for free. The list goes on and on…


The very nature of being open source means that the makers of software need to respect the community. Recently, Microsoft released Windows 8. Windows 8 brought with it a whole new user interface. This new interface hasn’t been a huge hit with the consumers, but Microsoft knows that it is free to do whatever it wants because we’re stuck with them. There is a lot of software that only works on Windows, and with Microsoft being the only legal source of obtaining Windows there’s not much we can do. It’s either Microsoft’s way or the highway. Not so with open source.

Any open source project has a third option: Fork. “Forking” an open source project means to make a copy of the source, give it a new name, and begin developing it on your own. Much like a fork in the road, the old project keeps going on its own, and the new project bears off into a new direction. Forking an application successfully is a very tall order. The old project is seen as the “definitive” version, with an established user base. To fork an application, you have to prove that your version is legitimate. Just because you forked a respected project doesn’t mean you get that project’s userbase; you must build your own user base. (this is what prevents people from “stealing” your project. If they fork your project, and the fork surpasses your version, that means they’ve added value you haven’t)

This happens often in open source. Sometimes the two forks coexist peacefully. Sometimes the fork dies, and the original regains any lost users. Sometimes the fork surpasses the original. However the cards fall, the community had its say; the better application won. This is a win for the community as well.

Back to the original topic, with this third option of forking, the developers can’t take their users for granted. At any time the userbase could revolt. Most likely, it’s not practical for you to fork an open source project. However, odds are that if your issue is real, others have it as well. Given enough unrest a fork is inevitable.


Lately all the talk in the news has been about NSA monitoring. The NSA has been collaborating with various software development firms to code methods into their software to allow them access. Even if you support what the NSA is doing this should concern you. After all, if the access point exists for the NSA, it exists for anybody. Given enough time, hackers will find these access points, and they will gain access.

This sort of thing cannot happen with open source software. Any change that gets made to an open source project is done in the open. Anybody can see the change, and the exploit has a very great chance of being discovered. There can be no back-door dealings with anybody if the house you’re coming in the back of has no walls.

What’s In It For Developers?

All that is great, but I still haven’t answered the original question: why would anybody do this?

Personal Gratification

Starting at the most basic, developers want personal gratification. Maybe they want to feel like they did something good. Maybe they want to feel part of something big. Maybe they want to stick it to the man. Maybe they want to give back to the community. Whatever it may be, it can be a powerful motivator.

Open Source As A Portfolio

Much like an artist, a developer needs a portfolio. Open source software can be a good way to demonstrate knowledge and experience with a programming language or framework. A developer could make a closed source application on their own time, but they wouldn’t be able to show it off for fear of “revealing their secrets”. Meanwhile, developing in the open, you could show a potential hiring manager specific examples of your work. Additionally, being an active contributor on a respected open source application can be a major boon to a resume.

Branching Out

Suppose a developer works at a company exclusively coding in Language X. Let’s call this developer “Brad”. Development is a fluid field; what is the big language today might be nothing a year from now. It is very important to not become stagnant. Unfortunately for Brad, he’s an expert in Language X, he’s barely touched up-and-coming Language Y. Even worse for Brad is the fact that his employer doesn’t care about his professional development. Brad’s job is to develop in Language X, and incorporating Language Y is a large risk that the financial people don’t feel is worth taking. Brad has no choice but to gain experience in Language Y on his own time.

Enter open source. Brad can find an open source project written in Language Y, and contribute. Brad could also start his own project in Language Y if he wanted. Either way, he’s got options. Development is a unique field in that one can reasonably “gain experience” on their own time. One can’t be a doctor, a lawyer, or a school teacher on their own time. But if Brad contributes to open source using Language Y, not only does he become more proficient in Language Y, but he has tangible proof of his experience in Language Y.

Similarly, a developer might have burned out on the technologies they use at work. Open source represents an opportunity to work on a project that excites them. The bean counters at work don’t care what you want to do, but nobody can tell you what open source project you have to commit to. A developer can contribute to a project that excites them, and they are free to move on at any time.

It’s Their Job

Shockingly, companies even pay developers to work on open source projects. For various reasons that I’ll get into later, companies use open source software. The direction of open source projects is often driven by those who contribute, and for this reason companies often hire developers to develop the aspects of an open source application that they care about.

But Think Of The Corporations!

Don’t worry, I’ve got them covered. As I mentioned in the last point, even companies care about open source. Even the tech giants like Google and Apple depend on and contribute to open source.

Using Open Source Libraries and Languages

Software companies often use open source libraries and languages. A library is a small piece of code that does a specific thing. It is not a full application in its own right, but can be critical to making a full application. Think of it like LEGO. If the individual LEGO blocks are a language, and the finished model is an application, then libraries are smaller bits made up of blocks. You assemble these smaller bits, then piece them together with some additional blocks to make the whole model.

Libraries are critical to making applications. Libraries also make up a large portion of available open source software. Libraries using restrictive licenses like GPL and friends don’t tend to make it into closed source software, libraries with permissive licenses like BSD and friends are often incorporated into closed source projects.

Similarly, programming languages themselves are often open source. This allows organizations to add features to a language to suit their needs. This sort of thing would not be possible with a closed-source language.

Outsourcing Development To The Community

Often, companies will open source products. This allows the community to scrutinize the software, adding new features, fixing bugs, and providing various other forms of feedback. This costs a company nothing, but can save it millions in development costs. The company can then use this feedback to improve the product. Typically, companies will open source components of a larger project that they sell. This allows them to crowdsource bug fixes and the like while still being able to sell their product.

They aren’t too concerned with the community forking their product because they still remain the “definitive” version, and there is little risk of them losing business.

Alternative Forms Of Income

Some tech companies just aren’t in the business of selling software. Google is one of the largest and most respected software development firms operating today, but they don’t actually sell software. Google is in the business of selling demographics data to advertisers. This puts them in the position of being able to freely leverage open source software to make their products. Google provides the google search engine, which is implemented using a variety of open source and closed source software. All of which is transparent to the users. Additionally, Google provides the Android Smartphone OS, which is based on the open source Linux operating system. Google provides countless other free applications and services using open source, and they can do this because these services allow them to glean more and more demographics data.

Similarly, Apple Computers is in the business of selling devices. Sure, Apple sells software, but most of their revenue is from selling various iDevices. Apple is also another huge contributor to open source. While Google uses open source to implement services (see point 1 in companies), Apple outsources development to the community. (companies point 2) Apple backs various open source projects, and incorporates them into their software. Mac OSX itself is a descendent of the BSD UNIX operating system, an open source operating system. A good example of this model is WebKit. WebKit is a piece of software used to read and display web pages, commonly referred to as a “rendering engine”. Apple is a major backer of WebKit, devoting many developers to it. When Apple deemed it ready, they incorporated it into their Safari web browser. Many other web browsers use WebKit as well, including Google Chrome. All largely thanks to Apple.

Another example of this is Apple’s open-source CUPS printing library. If you’ve ever printed something from a non-Windows computer, you probably have Apple to thank.

What Can You Do?

The final piece of this puzzle is: what can you do to contribute? Open source is a collaborative effort, and every little bit helps. However, being the lay person, you probably aren’t going to be contributing code or documentation. You can still help.

The First Step

The first step is to use open source. Personally, I believe that just using an open source project is contributing. By using it, you are legitimizing the project. You are a real person using this software, somebody actually cares! Given all the reasons I outlined above, this seems like a no brainer.

Secondly, you can spread the word. I’m not suggesting you be that guy who rants and raves about open source; frothing at the mouth about how you should use it or you’re stupid and a Bad Person, and support the “Evil Corporations”. No I’m suggesting that when given the opportunity to tell somebody about an open source application in a socially appropriate way, to rationally tell them about Open Source Project X, and that you use it because of [Logical Reason]. Is your friend frustrated with Microsoft Office? Tell them about LibreOffice. It’s that easy.

A Bit More Dedicated

If you’d like to do more, there are plenty of options. Many larger open source projects have statistics gathering features built into their applications. Things like Debian’s Popularity Contest, and Firefox’s Telemetry. Typically, these features can be disabled, however if you don’t mind a minor loss of privacy, you can considering enabling them.

Open source projects do different things with this data. Some use it to determine what kinds of people are using the software, and what things they use most often. This allows them to focus their efforts on the things that “matter”. Letting projects collect this sort of data allows them to improve the things that you use, and that you care about. This helps them because if they know what people want, they can better provide it.

Some open source projects might sell this data. Some might say this is an unacceptable breach of privacy (I count myself among those who would say that. For this reason I usually disable these features), but the fact is that they do this to get funding to improve the project. If you allow an open source project to sell your demographics data, it will make their data more valuable and help contribute to the quality of the software.

Either way, given the nature of open source, whatever information they are gathering from you is all out in the open for anybody to see. While you may not know what you’re looking at, there are people who do. Since typically only major open source projects do this sort of thing (minor projects don’t have a large enough user base to get any useful/valuable data), odds are almost 100% that somebody has independently reviewed it. A quick google search can tell you what they are gleaning and what it’s being used for. Personally, I take privacy very seriously, and I’d advise anybody to do a bit of research before enabling these sorts of features.

Another good way to help is by submitting bug reports. Nobody likes buggy software, but many bugs are nearly impossible to catch in testing. If you’re using an open source application and you find a bug, tell somebody! Ideally, the application comes with a built in automated bug reporting system. Poke around a bit, and see if there’s a “report bug” button. If so, give it a shot. Along with your message, these systems will send information about your configuration that will be useful to the developers.

Failing that, most projects have an easily accessable bug reporting system, usually this can be found on their website. Go ahead and submit a bug report. Try to be as detailed as possible, but if you don’t know something, don’t sweat it. Even if they can’t duplicate the bug, they know that something is wrong, and what general direction to look.

Giving Back

Finally, if you’re feeling generous, you could donate. I’m not going to name “worthy causes” here, but I shouldn’t need to. Do you have an open source application you really like? Donate to them. Maybe they have an online store, and you can get a T-Shirt or coffee mug. It really is that easy.

Profiling With Gprof

As promised, today I’ll be talking about GProf.

What Is GProf?

Gprof is a profiling tool. There are two versions: the original written for BSD UNIX, and a re-implementation for Linux. There are some minor differences between the two that you can read about here. We are going to concern ourselves with the Linux version.

Gprof is a pretty bare-bones utility. In the spirit of UNIX it does just one thing well. It doesn’t have any fancy menus or exotic extras. You just run it and it goes.

Preparing Your Program

First things first, we need to compile and link our program with profiling enabled. Using gcc, this is easy; just add -pg to the compile and linking options. If you’re using NetBeans, this is simple as well.

In NetBeans, right click your project and click Properties

Click Build > C Compiler. In the Additional Options field, add -pg to the end.

Click Build > Linker. In the Additional Options field, add -pg to the end.

Click OK. You are now good to go. Clean and build your project and you are ready for the next step.

Run Some Test Cases

Here is where things get confusing. You’ve compiled your program, surely you have to run your application within gprof similar to how you ran Valgrind, right? Let’s give that a shot…

# gprof [your_program] gmon.out: No such file or directory

Hmm… I’ll save you some time: Google will tell you that gmon.out should be there, clearly you messed something up and you’re a bad person.

Here’s the secret: you aren’t supposed to run your program through gprof at first. You need to run your program by itself first and it will create gmon.out in the current directory upon termination. Repeated runs will replace the existing gmon.out file. Run your application and perform some test cases. Follow good test practices and really stress your program. Now, you are ready to generate your profile data. Make sure gmon.out is in the current working directory and then…

gprof [your_program] > [output_file]

GProf should return without error and there should be a new file. Open it up in your favorite text editor. As you can see, it contains a flat profile, and a call graph of your program. Below each is a description of the columns. You should be able to take it from here, but if you want additional information check out the GNU gprof webpage.

Oh yeah, and don’t forget to remove the -pg from your compiler and linker lines when you’re done; they’ll tank your performance.

Bringing The Portability With GModule

As I’ve been writing DMP Photo Booth, I’ve been taking great pains to improve portability. I’ve got a fancy module-based architecture designed to segregate the non-portable sections of the project from the Core. The only problem? A bunch of ugly POSIX calls: dlopen(), dlsym(), and dlclose(). Kind of defeats the purpose of using modules for portability if I don’t load said modules in a portable way, doesn’t it? I thought so too…

Enter GModule

GModule is part of the GLib family of libraries. GModule provides a portable way to handle working with shared libraries. It works on any POSIX compliant platform, as well as Windows, and HP-UX via its shl_load() mechanism. You can read more about GModule in the GLib Reference Manual. While I’m sure there is some edge case that GLib doesn’t cover, this is far more portable than I’d initially envisioned DMP Photo Booth being. (Yay, HP-UX support!)

Another consideration in all of this is the adding of dependencies. However, since I’m already using GTK3 for my GUI, I already have a GLib dependency, so there is no added burden to using GModule.

The How

GModule is actually quite similar to using POSIX dlfcn.h functions. Some semantics are different, but GModule has functions that are roughly equivalent to the POSIX functions.

GModule * g_module_open(const gchar *file_name, GModuleFlags flags);

G_module_open() is the replacement for dlopen() in POSIX. The GModule pointer that it returns is the replacement for the void pointer returned by dlopen(). GModuleFlags is an integer flag that can be boolean or’d in. Your options for this are G_MODULE_BIND_LAZY and G_MODULE_BIND_LOCAL which are equivalent to RTLD_LAZY and RTLD_LOCAL.

gboolean g_module_symbol(GModule *module, const gchar *symbol_name, gpointer *symbol);

This is your replacement for dlsym(), and functions mostly in the same way. module is the gmodule pointer to extract symbols from, symbol_name is the symbol to get, and symbol is the function pointer to populate. This function returns true if successful, and false if not. This function is commonly called like this:

if (!g_module_symbol(dmp_pb_camera_module, "dmp_cm_capture", (gpointer *) &dmp_cm_capture)) { /* error handling here */ }

This idiom can be found throughout the GLib documentation. Did you see the craziness that is argument number 3? Dmp_cm_capture is a function pointer, as you may remember, but GObject tends to make things a little tricky, and will throw thousands of warnings if you don’t cast your function pointer to a gpointer *. The definition of gpointer is:

typedef void* gpointer;

That means that a gpointer * is actually void **. Therefore, you are expected to pass in the address of your pointer using an ampersand, hence (gpointer *) &dmp_cm_capture.

gboolean g_module_close(GModule *module);

As a nice change of pace from the last function, this one is quite straight forward. You pass in your GModule pointer, and it closes it, just like dlclose(), then returns true (or false, if an error occurs). Nothing particularly noteworthy here.

gboolean g_module_supported();

While this function does not line up with a POSIX function, I felt it was important to mention this one. This function returns true if you are on a platform that GModule supports. If you’re on Linux/UNIX/OSX, or Windows, or HP-UX, this should return true. If you’re hacking your Atari-2600, it will most likely return false. Stick this in front of calls to GModule functions and save yourself a headache.

Moving Forward

My design philosophy for DMP Photo Booth is that the Core should compile on any (typical) platform with no changes. Using GLib, this seems to be within reach. With GModules, GTK3 User Interfaces, I’ve kept the faith so far. Looking through the GLib documentation, it has GThreads, so when I inevitably get mired in threaded programming, I should be golden. GLib also has support for Pipes and file IO via GIOChannel, but the documentation claims only partial support in Windows.

Here’s hoping all goes well!

The Case of the Missing Tip

For the past 10 years or so of my career, I’ve mostly been employed as a sysadmin. During this time, I spend a great deal of time working with Solaris 10. Solaris 10 has a command called tip. Tip can be used for basic serial communications. You basically just pass it a baud rate, and a serial device, and it connects the STDIN and STDOUT.

Working with Arduino, sometimes you want to connect to the serial port and communicate with it. There is a whole Serial library for reading from and writing to Arduino via the usb cable. The Arduino IDE also comes with a serial monitor for use. However, using NetBeans, I find it kind of silly opening Arduino IDE just to use the serial monitor. Unfortunately, tip seems to be a Solaris-only utility. At a loss, I turned to google.

Google Proves Unhelpful

Shocking, I know. The number one solution to this is “Well, just write something in C!” I know what you’re thinking, and yes it did occur to me that this would be right up my alley. However, I know there is a utility that exists that does exactly what I want. In the interest of not re-inventing the wheel, I search on. The number two solution to the problem is a program called Minicom. Minicom is a fairly heavy menu-driven program that does serial comms. From what I can tell, it seems to be more like Microsoft’s Hyper Terminal than tip. Since I’m just looking for a simple command line utility, I move on.

Finally, after much searching, I come across my solution. There is a program called cu that seems to function exactly like tip. It even share the same disconnect code: ~. to hang up a connection. Even better: it’s in the repositories!

On an Ubuntu system, all you need to do is:

sudo apt-get install cu

…and you’re good to go. To connect is simple, just enter:

cu -l /dev/ttyWHATEVER -s BAUD_RATE

…and you’ll connect to your Arduino. To hang up, just enter ~. and it will disconnect (Ctrl+C and Ctrl+D have no effect).

In Defense of Multiple Partitions in Linux

If you’ve ever installed Linux or UNIX, you’ve surely come across this idea: installing across several partitions. These days it’s not so common in many distros, but just a few years back everybody wanted you to do it. Reccommended partition tables included: /, /var, /var/log, /usr, /boot, /home, /tmp, SWAP, and /everyone/and/their/mother. The silliness knew no end. Lately, in the age of Ubuntu, distros have decided that this list could be reduced to: /, /boot and SWAP. I’m here to urge you not to go along with this.

Why not?

I’m glad you asked. If you do a quick google search for why you should have multiple partitions like this, you’ll see the usual enterprisey reasons. The words “security” and “disk quotas” will come up often. You might think that as an end user you don’t really care about these things. You’d probably be right. Indeed, in this day and age you tend to only see these crazy partitions schemes server operings systems like Solaris.

There is one thing that typical Linux end users do tend to do though: switch distros. With so many choices, it’s easy to want to try a different one. Maybe you’re tired of replacing Iceweasel with real Firefox, so you switch to Ubuntu. Maybe Mark Shuttleworth spoke ill of your mother, so you switch to Mint. Maybe you want real Java, which only comes packaged in a .rpm, so you switch to Fedora. Either way, if all your filesystems are in one partition you’ll be sorry.

One of the historical problems with Linux that has yet to be solved is cleanly upgrading and replacing distros. Some try to support in-place upgrades, but the process still involves prayer. Others have extremely convoluted processes that are much harder than starting from scratch. My personal home server is still running Debian Squeeze because the upgrade process sounds like so much work.

Oh Dear. What Should I Do?

The solution is simple. There is no need for the complicated partition schemes of yore. All you need to do is partition your system like so: /, /boot, SWAP, and /home. If you partition your system like this, then switching distros is simple. All you need to do is not touch this partition. You will be able to delete and recreate /, /boot, and SWAP, and leave /home untouched. This will ensure your data is safe. Obviously backups are in order when messing with partitions, but it’s a pretty safe bet if you decide to gamble on it. So long as you don’t use some crazy filesystem for /home, the new distro should be able to mount it seamlessly.

%d bloggers like this: