Posted by: Vino
Date posted: Oct 22 2004
User Rating: 5 out of 5.0 | Number of views: 12006
Number of comments: 2 |
Description: How to build an old-school gcc 2.x binary on a gcc 3.x system. |
It stumped me for the longest time, but finally I figured out how to get Half-Life's server to compile and run on Linux boxes worldwide. There are a number of issues associated with this that we first have to look at:
-- End of line style discrepancies
-- ANSI compliance
-- Binary compatibility between gcc 2.x/3.x
-- Standard C library linking
-- Architectures
-- Forward regression... or something
This tutorial goes beyond the remedial and outdated instructions that Valve gives you in the SDK to solve some of the more interesting issues that HL modders are faced with when compiling their Linux distributions.
To start out with, you must first have a Linux box. Your Linux box can be that old Compaq in the attic, it doesn't matter. Linux runs on anything. (Actually, you'll have to have a 386 machine to compile the HLSDK unless you want to cross-compile... but if you know how to do that, why are you reading this?) Dig it out and stick it next to your current computer. You don't even need a monitor, because through the magic of putty and SSH, you can use Linux without even having an extra keyboard. There are many different Linux distributions, and the choice of which one to use can be daunting. Allow me to simplify it for you: if you've never used Linux before, you should shoot for Redhat or Suse. However, if you have Linux already, may I suggest Gentoo?
In any case, this is not a tutorial for how to set up a Linux box, there are already plenty of those on the internet. So, I'll just get straight to the compiling issues. When you have your SDK on your linux box, navigate to the dlls directory and use the "make" command to (try to) build the game.
EOL STYLE
Unix-based systems and Windows systems use two different ways of describing the end of a line (EOL) in a text file. Windows uses CRLF (carriage return/line feed) and Linux uses LR (just line feed). So, while a text file with Windows CRLF might look like this:
| | | something\r\n
or\r\n
other\r\n |
that same text file with Linux LF will look more like this:
Of course, the \r\n and \n characters are totally transparent to the typical user, but to a program written on either platform, there is a big difference. If the program is designed to recognize one type of EOL style, a program of the other type will appear misshapen and ugly in a text editor. For example, I'm sure we've all opened a file from the internet in Windows Notepad to discover all these large black boxes and the entire document on one line. This occurs because Notepad doesn't recognize Linux's LF EOL style, so it clumps everything up on one line and sticks those black boxes where the new lines should be. Conversely, if you were to open up a file with Windows EOL style on a Linux box, you might see that the end of every line has a ^M character.
Now, most modern programs automatically convert newlines to the style of their native OS if they detect them. GCC 3.x does this, and so do all of Microsoft's tools, so you won't run into this problem as much nowadays. However, GCC 2.x does not do this conversion, and if you try to compile a program with GCC 2.x that has a Windows CRLF style, it will spit out craploads of garbage in error messages. So, you need to convert from CRLF to LF before you can use GCC 2.x to compile your server.
You can actually find a description of this problem in your SDK in the file called linux_compiling.txt, it suggests a number of solutions for the problem. It says that one way to fix this is to transfer your programs to your linux box using a good FTP client that will do this for you. Another way to do it is to use the dos2unix program (Gentoo users can emerge this) to convert your files. Also, for a quick convert, most Linux text editors these days (nano/pico vi emacs) will do the conversion for you when you open the file. However, all these methods are inferior to the my favorite way of doing this, which is allowing your version control system to do it for you.
WHAT YOU SAY?!? You aren't using a version control system? Well, I feel sorry for you. Every person working on a major project that is larger then a thousand lines of code should use a version control system to organize his code, even if he is working on the project by himself. In any case, the version control system I use, called Subversion (which is the best in my opinion, and free) does this job for me transparently (TWL has a tutorial on Subversion here -Ed.). You can also use CVS if you are suicidal, I mean inclined. In any case, your version control system will convert your entire project to Linux-style EOL format whenever you update the project.
You can verify that your files have the correct EOL style with the 'file' command.
| | | $ file world.cpp
world.cpp: ASCII C program text, with CRLF line terminators
$ dos2unix world.cpp
dos2unix: converting file world.cpp to UNIX format ...
$ file world.cpp
world.cpp: ASCII C program text |
ANSI COMPLIANCE
HLSDK 2.3 (or any version) is not ANSI-C compliant, but GCC 2.95 is. What this means is that GCC 2.95 and Microsoft's compilers interpret C/C++ code differently. Thus, a project which compiles cleanly with no errors on a Microsoft compiler may have errors on a modern GCC compiler, and the other way around. Thus, you need to find a version of the code that both Microsoft's compilers and the GCC compilers agree with. Porting the SDK to be ANSI-C compliant is really not difficult, but it is rather tedious. I'll give here a couple pointers to demonstrate some of the things that need to be done. These are some typical errors that you'll find while compiling the SDK:
COMPILER ERROR:
| | | effects.h: In method `void CSprite::AnimateAndDie(float)':
effects.h:82: static_cast from `{unknown type}' to `void (CBaseEntity::*)()' |
SDK CODE:
| | | | SetThink(AnimateUntilDead); |
ANSI-C CODE:
| | | | SetThink(&CSprite::AnimateUntilDead); |
COMPILER ERROR:
| | | | weapons.h:448: ANSI C++ forbids declaration `HasWeapon' with no type |
SDK CODE:
| | | | HasWeapon( CBasePlayerItem *pCheckItem ); |
ANSI-C CODE:
| | | | BOOL HasWeapon( CBasePlayerItem *pCheckItem ); |
COMPILER ERROR:
| | | animation.cpp:337: name lookup of `i' changed for new ANSI `for' scoping
animation.cpp:332: using obsolete binding at `i' |
SDK CODE:
| | | for (int i = 0; i < pstudiohdr->numbonecontrollers; i++, pbonecontroller++)
{
if (pbonecontroller->index == iController)
break;
}
if (i >= pstudiohdr->numbonecontrollers)
return flValue; |
ANSI-C CODE:
| | | int i;
for (i = 0; i < pstudiohdr->numbonecontrollers; i++, pbonecontroller++)
{
if (pbonecontroller->index == iController)
break;
}
if (i >= pstudiohdr->numbonecontrollers)
return flValue; |
COMPILER ERROR:
| | | effects.h:171: object-dependent reference `SUB_Remove' can only be used in a call
effects.h:171: to form a pointer to member function, say `&CBaseEntity::SUB_Remove' |
SDK CODE:
| | | | inline void LiveForTime( float time ) { SetThink(SUB_Remove); pev->nextthink = gpGlobals->time + time; } |
ANSI-C CODE:
| | | | inline void LiveForTime( float time ) { SetThink(&CBaseEntity::SUB_Remove); pev->nextthink = gpGlobals->time + time; } |
If you feel like saving yourself some time, you can search botman's site on planet half-life and find some old patches he made to convert the HLSDK to ANSI-C compliant, then all this will be as easy as the patch command.
BINARY COMPATIBILITY
When Half-Life was made back in '98, Valve used a compiler called EGCS to compile HL, which is a branch of the GCC 2.x line. This was the latest and greatest Linux compiler at the time. However, since then, the Linux world has progressed to bigger and better things. At the time of this writing, the most recent GCC version is 3.4.2. All of the programs compiled with Linux today are compiled with this version.
Unfortunately, we can't use GCC 3.x to compile our Linux .so's because of something called binary compatibility. GCC 2.x and GCC 3.x binaries are not compatible with each other. Between versions 2 and 3, the developers of GCC had to change the format that the binary uses in order to make more improvements to the executable. The result is that while GCC 2 and 3 programs can coexist on the same Linux system, they cannot be linked together.
All of the binaries provided by Valve (ie hlds_i486 hlds_i686 hlds_amd etc) were compiled with gcc 2.x, so in order to have our server .so work with them, we also need to compile it with a legacy GCC 2.x compiler. The HLSDK tells us to use EGCS, but the latest in the line of 2.x compilers is GCC 2.95.3, so we will use that. You can find this version of GCC on the GCC 2.95 website, but chances are that you use RedHat Linux and the RPM system. If this is the case, you can likely find RPM's for GCC 2.95 on your installation disks. They will be named something like compat-gcc-2.95.rpm or similar. GCC 2.95 should compile cleanly on most GCC 3.x systems. I use Gentoo Linux, and I simply downloaded GCC 2.95 from its website and compiled it myself.
When you have installed GCC 2.95, you will need to make sure that you are using it instead of GCC 3.x that you currently have. To do this, go into the Makefile and look for this line:
"gcc" points to the current compiler on your system which is likely GCC 3.x (or else why are you reading this article?) It needs to look more like this:
This will specify that you want to build the HLSDK to with the legacy GCC instead of GCC 3.x.
STDLIBC++ STATIC LINKING
By this point you should have an SDK which you can compile correctly. However, it may not run at all. You can stick it in your mods dlls directory and try to boot it, but it just says "Segmentation fault" and exits. This may be because your system is loading up the standard C library built for GCC 3, which is binary incompatible with your new .so file. Not only that, but suppose a person running a FreeBSD server decides he wants to run your mod? Technically it is possible, but I guaruntee you it won't work at all, because the standard C library on a FreeBSD system is very incompatible with the one on a Linux box. Thus, unless the standard C library installed on your target machine's box is compatible with the one on the box you compiled the binary on, your game will not boot.
The solution to this is to statically link the GCC 2 standard C library with your program. This way, no matter which Linux distro your game is installed on, it will always carry with it a version of the standard C library which is compatible with it. To do this, first you need to find the GCC 2 version of the standard C library.
| | | $ locate libstdc++.a
/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.3/libstdc++.a
/usr/local/gcc295/lib/gcc-lib/i386-pc-linux-gnu/2.95.3/libstdc++.a
/usr/local/gcc295/lib/libstdc++.a.2.10.0 |
On my computer, gcc 2.95.3 was installed into /usr/local/gcc295 but yours may be installed in a different place. You are looking for a file named libstdc++.a in your gcc 2.95 directory, NOT the one in your gcc 3.x directory. Once you find the path to it, you need to add it to your Makefile. Find the line in your makefile that looks like this:
You need to tell your linker to link this file in with your game, and you can do it like this:
| | | | LDFLAGS=/usr/local/gcc295/lib/gcc-lib/i386-pc-linux-gnu/2.95.3/libstdc++.a -lm |
You can also use the "nm" command to see which external functions your program needs and which version of glibc is linked in:
ARCHITECTURES
The land of computers is strewn with incompatibilities of all kinds! For example, if you have a Pentium 4, your processor is an "i686" processor, but if you have an old Duron, your processor is a "k6". Programs compiled for AMD or Pentium processors are often incompatible with each other. However, all processors that run Half-Life will run programs compiled for an i386, i386 is the lowest common denominator of consumer processors. When you compile your project, it will compile it for an i386, so that all processors can run it. However, it is standard practice to compile and distribute different versions of your game that run on specialized systems. The idea is that a program compiled for an i686 will run faster then a program compiled for an i386 on that compiler. Fortunately, with the standard Makefile provided in the SDK it is really easy to do this. First, you need to go into your Makefile, and replace all the instances of "-m486" with "-mcpu=$(ARCH)" and then you can do this:
Using this command, you can override the ARCH value defined in the Makefile to whichever one you desire. However, it is vitally important that you do a "make clean" between each one to wipe out the object files compiled previously. If you're a pro and you want to give each architecture its own separate object file directory, you can make a simple modification to your Makefile, find the lines that look like this:
| | | DLL_OBJDIR=$(DLL_SRCDIR)/obj
WPN_SHARED_OBJDIR=$(WPN_SHARED_SRCDIR)/obj
PM_SHARED_OBJDIR=$(PM_SHARED_SRCDIR)/obj
GAME_SHARED_OBJDIR=$(GAME_SHARED_SRCDIR)/obj |
and make them look like this:
| | | DLL_OBJDIR=$(DLL_SRCDIR)/obj-$(ARCH)
WPN_SHARED_OBJDIR=$(WPN_SHARED_SRCDIR)/obj-$(ARCH)
PM_SHARED_OBJDIR=$(PM_SHARED_SRCDIR)/obj-$(ARCH)
GAME_SHARED_OBJDIR=$(GAME_SHARED_SRCDIR)/obj-$(ARCH) |
This will cause make to generate a separate directory for all of your architecture distributions, and you will no longer have to make clean between each one.
When linking stdlibc++, the part of the path that says "i386-pc-linux-gnu" is very important, because it lists the processor architecture that this version of stdlibc++ was compiled for. If your libstdc++.a is of type i686 or something else, then it may fail to run on other architectures, for example amd or k6. You need to compile your libstdc++.a as an i386 to ensure maximum compatibility over different architectures. If you want to be really daring, you can compile a separate version of libstdc++.a for each architecture you want to distribute for.
FORWARD REGRESSION... OR SOMETHING
In this final section I will look ahead to how this article will apply to Half-Life 2. The game just went Gold this week, so I don't suppose anybody will be using this tutorial unless I make it apply to HL2. We were just informed by Alfred on hlcoders that the Linux HL2 binaries have been compiled with GCC 3.4.1, so lets look at the ramifications of this.
-- EOL Styles
EOL styles will no longer apply to compile Half-Life 2's source code, since GCC 3.x correctly handles Windows-style CRLF line endings.
-- ANSI Compliance
I have yet to see the HL2SDK, but from the way Alfred is talking I trust that Valve has made it ANSI-C compliant. Thus, the porting issues will also no longer apply.
-- Binary Compatibility
We will no longer have to keep legacy versions of gcc around in order to build a working version of our games on Linux. However, according to GNU's tentative release timeline, GCC 4.0 will be ready for release in early 2005. This means that all us Linux users will be migrating over to it, while the HL2 server binaries will remain at 3.x, that is, unless Valve recompiles them. Otherwise, since a major GCC release will once again break binary compatibility, everything in the Binary Compatibility section above will still apply if you simply replace GCC 3.x with 4.x and 2.x with 3.x, and we will all be compiling "legacy" versions of GCC 3 to compile the HL2SDK. Isn't the future great?
-- stdlibc++.a Linking
This section will always apply. Linking stdlibc++.a is a great way to ensure that your game will run on every distribution of Linux.
-- Architecture Variations
This section will apply moreso in the multi-architecture world of Half-Life 2. GCC 3's options for architecture compatibility are even more diverse then GCC 2's, and with the world of 64 bit not too far off, we will have multiple platforms for which to compile things. |
|
User Comments
Showing comments 1-2
|
this helped me alot :) thx |
|
You must register to post a comment. If you have already registered, you must login.
|
297 Approved Articless
8 Pending Articles
3940 Registered Members
0 People Online (1 guest)
|
|
