Perils of strdup

Let’s take two nice RPM based distros: Fedora12 and CentOS5.5. And the following piece of code:

char *s = strdup(something);

This works on Fedora. All the time. This works on CentOS too … most of the time. So, pale C programmer, be aware that using strdup on CentOS is evil, evil, evil. Instead do this:

int l = strlen(something);
char *s = (char*)calloc(l + 1, sizeof(char));
if(!s) return;
memcpy(s, something, l);

This works all the time… Some people say, that strdup is not standard C. Fine with me. But it still should work once it compiles.

Pitfalls of Apache mod programming

I think apache needs no introduction to anyone having some knowledge in Linux, web servers, internet or other digital-era inventions … Also, if you’re a little bit more tech-savvy, you might know, that you can write apache modules, which can do various other things beside of the standard “give me the file” feature already existing in apache. A very good tutorial can be found at: http://threebit.net/tutorials/apache2_modules/tut1/tutorial1.html this will introduce you to the basics of how to write an apache module, starting from almost zero, and if you are more interested you also can read some of the chapters from the great book of Doug MacEachern and Lincoln Stein: “Writing Apache modules with Perl and C”.

On the other end writing apache modules is a pretty cumbersome task, so there are not too many resources available. I will try to share a few of my “Lessons Learned” throughout this blog with you, in case it will help someone.

Lesson 1. – Creating a web server inside an apache mod.

Let’s suppose you have the weird situation, that you need a module which needs to act as a server (yes, don’t forget, you are already integrating in an existing solution) so that clients can connect to it, and exchange information. This might freak out a security expert, but let’s hope there’s not even one around to see this requirement. The first thing you need to take into consideration, is that apache doesno’t really like the standard BSD socket functions. They work, but in my experience they are totally unreliable, since their resources might not be manageable by the apache runtime… (yes, apache is managing its resources in so called pools, meaning, you don’t need to take care of memory management if you decide to use these… and I highly recommend using these apache pools. They’ll save you a lot of time). To get your hand on the apache portable runtime’s socket management functions please consult the following link: http://dev.ariel-networks.com/apr/apr-tutorial/html/apr-tutorial-13.html

This is the best that is out there right now, so just read it and understand it. But your problems are far from being over. Remember the following: by default, apache is allocating a process to a request, because on the posix flavored world of X-es, spawning a process was much easier than spawning a thread. So, apache gets the request from the remote browser, spawns your process, your process loads the requested modules, your process gets the request, does some work on it and the result is sent back to the client, the process dies. Let’s see what is happening for your module: when apache spawns the new process there are some callbacks executed, usually these register some so called hooks, that are executed in various phases of the request processing. More details can be found out at: http://httpd.apache.org/docs/2.0/developer/modules.html

Usually you would like to create the web server in the initialization phase of the module and would like to keep it there till the apache is up and running. But with the default configuration, a web server is created for each of the processes, and with some network programming background you easily spot the bug: there can be only one web server on one port. Period. The solution is that you configure apache to run in a multi threaded, one process setup, this way you can create a shared web server between the various apache threads as a global, static variable.

Now you’re happy. You deploy your code, works flawlessly… for 3 or 4 hours. Then suddenly crashes. Lots of debugging don’t bring you closer to the solution. You read, and read through the code. Put in lots of debug messages… re-deploy. Great, works for 7-8 hours, then it crashes again. Yup… multi threading, sharing a global object is usually a bad idea. You deploy some cleverly placed mutexes. Finally, your code works as it’s supposed to. Don’t forget to use apache’s own mutex locking mechanism

How to write C++ applications under Windows – Chapter 1

How to compile C++ applications under Windows

A friend of mine has asked me how to compile C applications under Windows. He just started walking the pathways of C programming, so obviously some questions might come up during the rocky road of C development. To help him I’ve compiled this short list. I want to stuck to sources which are freely available on the internet, and are easy to start up with.

Here is the first application I want to compile


#include
int main(int argc, char** argv)
{
printf("%s was called with %d arguments\n", argv[0], argc);
}


So, here are my methods of compiling a C program under windows.

Visual Studio Express 2010

The fastest way to start C development under Windows is to download the freely available Microsoft Visual Studio Express: http://www.microsoft.com/express/Downloads/#2010-Visual-CPP . After the download has finished, run it, it will download more stuff, but after the mandatory restart you will end up with a fully functional IDE, which will be pretty easy to use.

When it’s set up just start it, and go to File -> New -> Project (or just press Ctrl + Shift + N) and select Win32 Console application, down below fill in the required fields and press Ok. The screenshot looks like:

Now, on the next screen do the following: Click on the Application settings, and check the empty project.

Screenshot:

Press Finish.

Now you have an empty project. I personally prefer creating empty projects, since you have a much better control over file creation.

The next step is to create a C++ source file and add it to the project, so: right click the project icon select Add -> New Item…

Screenshot:

in the new dialog box select select C++ file.

Screenshot:

And click Add…

type in the program above, Save and press Ctrl + F5 to run it. The result should look like.

Now, this should do it.

The advantages of the Visual Studio approach are that you will get a very good IDE, and you will have the chance to familiarize yourself with it, later it might come handy. This is an industry standard application, if you will get a job later in the Windows application development area, the chances that you will use Visual Studio are almost 100% otherwise do not work for that company.

The disadvantages Visual Studio are, that you will get used to it … and later when you will not have the chance to work with it you will really miss all its great features. Beside of this, no disadvantage was identified.

The next chapter will present how to write applications using different IDE’s

blackmagic in linux

So, I have this wonderful blackmagic SDI card, insert the card in its PCI-E slot, compile the drivers, insmod and then my machine freezes… Restart, insmod, freeze. Restart, install official driver from their site. Restart. Not even starting, kernel panic in the first screen… some googling, and then back to the provider site. They tell me is I want to use pcie_aspm=off in the bootloader. Good… card remove, edit /boot/grub/menu.lst append stuff at the end, card back, start machine and it works. Wonderful…

ffmpeg and VP8

After some struggle I managed to compile ffmpeg with Google’s new vp8 codec (libvpx implementation) (by struggling I mean, I had to recreate some typedefs in the VP8 code and change some structure names in the ffmpeg source, but generally it was quite straightforward making it work… just follow the compiler error messages, they guide you to the specific location … and of course, don’t forget to add –enable-libvpx to the configure parameters. I am using fedora12, maybe other platforms don’t have these problems because it’s quite unrealistic they’ve released in this buggy conditions).

However it feels a little bit slow, so there’s more work to be done on it. But what I really like about it is the quality of the picture. It’s good, almost close to the original one. Also, the size of the file is acceptable (a 75 second clip was transcoded into 4.6 MB and the same clip was transcoded to 8.2 MB using theora with a quite low quality video (just used default settings)).

XP 2010 Day four + conclusions

The fourth and last day of the conference was again dedicated to workshops and tutorials. In the first half of the day I participated to a session with the not so promising title “Building software that matters” however it turned out to be more interesting than I have anticipated. Basically it was a big brainstorming, where all the participants have shared their experiences and best stories in their software development career till time and then all the others have tried to come up with a solution (or at least some suggestions) to the situation and how they would have solved it.

The second session of the day was about “serious games”, ie. team building games. We were divided into small teams, and in the team one observer was chosen who was monitoring the team. The role of the team was to decide the priority of features to be implemented in various iterations of an agile development project. The name of the game was “Prune the project tree”, more details at Cory Foy’s web page. It was interesting observing the team evolve and get into conflict with themselves based on their personal priorities and likes.

And some conclusions at the end:

1. There were good and bad presentations. You really had to filter out the program and make a good selection of the offering to end up with something. Generally a good tip: go to the sessions that most geeks attend, and there’s a good chance you’ll learn something new. If not from the presentation, then from the questions and replies afterwards. Otherwise you’ll end up at a boring academical presentation which only 2 people on the planet understand. One of them is the presenter.
2. An interestingly sounding title by a proeminent person in the domain with multiple books behind his back can cover a really boring and factless presentation and an average IT guy (plain programmer) can give a more interesting and useful session … or at least for programmers.

XP 2010 Day three

The day started with two very contradictive sessions about what to do with your legacy code, and when to re-start from scratch an application. The first speaker convinced us, that regardless that your codebase is ugly, unreadable and hard to maintain, it’s more worth investing two months of time in refactoring it, than throwing it away and restarting from scratch with a brilliant architecture. The reason is simple: as long as you are rewriting your entire application, you cannot respond to market changes in your application since you don’t have an application, so the concurrency will “eat” your lunch.

The second presentation on the other end tried to convince us of the following: it’s worth rewriting your application from the scratch, but you have to do it wisely: do not put the old code to rest from the first day, but write code that duplicates the functionality of it, and let the two systems run parallel for some time, and slowly-slowly migrate the customers from the old system to the new system… basically this should do it in a very transparent way.

The third presentation was about practical testing and tools. The presenter introduced a few testing frameworks for web applications (for example: acunetix for vulnerability scan, or selenium with the WebDriver API for unit testing WEB pages using Java or using the cucumber framework for testing ruby applications).

The fourth presentation was about a theoretical approach about how the continuous run of unit tests in the background can actually contribute to a more thoroughly tested application. Of course, this needs to have someone who regularly updates the unit tests…

Beside of these there were various short sessions and discussions which have presented mockito, how to test more easily with FEST-asserts, etc… (and there was a great free for all food buffet, with various local and not so local food… no wonder the Rica hotel has the best breakfast in Norway. It was delicious  )