Welcome, Guest! Login | Register

Using Source Control Management In Mods [Print this Article]
Posted by: Persuter
Date posted: Aug 22 2004
User Rating: 4.2 out of 5.0
Number of views: 7682
Number of comments: 10
Description: This article will describe how to get Subversion, an SCM server, and TortoiseSVN, an SCM client, up and running, and how they can help your mod work better and more efficiently.
What Is Source Code Management?

Source code management, or SCM (also called "version control software"), allows many developers to work on the same piece of software at once. A central server (called a "repository") maintains all the code, keeps track of who's working on what source files, and attempts to integrate changes to the same piece of code. This allows for many different helpful features. One is called "version management", in which the repository maintains a history of exactly what's been done to each file at what time. Thus, I can download the code base EXACTLY as it was at, say, 10:33 PM on August 14th. This is very helpful when the code base breaks and you don't know exactly what broke it and don't have time to figure it out -- just back the code up.

Another is "branching". Let's say that your mod project has just released version 1.0, and you've started working on 2.0, which is a major revamp of the code. However, you're still going to want to continue to release bug fixes and patches for 1.0. No problem -- tell the software to create a "branch" of code called 2.0. Now you basically have two copies of the source code, one for 1.0 and one for 2.0, and can work on either or both of them without worrying about changes affecting one or the other.

A third is that there's a bunch of completely free programs that you can download to do SCM, and a fourth benefit is that not only will this allow your programmers to easily maintain a single code base, artists can also use SCM to provide the latest textures, maps, and models to everyone easily and in a centralized place. It also makes it so that getting a release together is as simple as clicking a single button, waiting for the files to update, and then zipping it up. No muss, no fuss, no chasing down artists or coders for that final version -- if they've checked it in, it's there.

How Does Source Code Management Work?

To actually get the files, you do what's called an "update". The server checks what files aren't up-to-date on your computer, and downloads the newest version of those files to your computer (actually, it only downloads the changes, your computer updates the files correctly). No more downloading an entire SDK even if only a few files have changed. No more downloading a hundred megs of object files that your idiot teammate left in the ZIP file (I've done it a lot :D). Again, you don't have to update to the latest files. If you only want to get the latest for some files, it'll do that. If you want to get the files that were current at a certain time, or at a certain release, it'll do that. Basically you can set your SDK to any state that it was ever in.

Once you've gotten your files and edited some of them, you then "commit" the changes to the repository. This means that the repository will check what changes you've made to what files, and if no one has also changed those files in the interim, then it saves the changes and from now on, anyone who gets files from the server will get your changes. (Note that I said that it saves your "changes", not the actual file. A file is built up through many changes (often called "diff"s, from the UNIX program that compares files and was used extensively in early SCM).) If someone else has changed those files, you'll have to "integrate" them. Sometimes the server can do integration for you, if the two changes don't conflict with each other. In the quite rare event that you find you've updated the same code someone else did, integration is made painless and simple. And honestly, if you're working on the exact same part of the code that someone else is, it's important for a human to be putting together the code.

Also, when you commit the files, you can add a "commit comment" which lets the other people know what you've done. An example of a commit comment might be "Fixed bug where gun was deploying in frame 4 of its animation". That way, when people are checking what you did to the code, they can easily see what you were trying to do.

Who Can Benefit From SCM?

All mod teams will benefit from SCM, period. I use SCM when I'm working on projects entirely by myself. Once you really get used to SCM, you'll find the old way you were doing it messy, dangerous, and annoying. Instant reversion of files alone makes it worth it (I edit a lot of files I shouldn't :D).

I'm Sold! How Can I Implement It?

Downloading Subversion
There are many different SCM packages out there. I am recommending Subversion, a much-used and free SCM package, for the server side (the repository), and TortoiseSVN for the client side (your developers). Subversion is stable, useful, and has various features that CVS, which is probably the most well-known and oldest free SCM package, doesn't. TortoiseSVN integrates with your computer very nicely. The rest of this article will assume that you are going to use Subversion and TortoiseSVN.

Now, you'll need a computer to run the Subversion software. It's very lightweight and the bandwidth is basically negligible unless you've got about a thousand developers working on something. Thus a home computer is perfectly acceptable to run it on, and that's what I'll be showing you how to set up. I will be assuming that you know the IP of the computer you're connecting to. However, in these days of DHCP, that's probably unlikely. Thus you'll probably want to use a free domain aliasing service like DynDNS.com. However, this isn't a tutorial about DynDNS so I'm going to skip that.

I'm further assuming that this home computer is going to be running Windows. Since that's what most home computers run these days, I feel pretty safe in assuming that most of my readers have at least one lying around the house. However, setting it up for Linux or some other POSIX system should be no trick. Check here for your preferred operating system (unless you're running Windows, in which case continue on). It would probably be much more helpful to run Subversion on your web server using its Apache 2.x interoperability mode -- unfortunately I don't have a web server to do it on. :) As I say, though, a home computer will be more than adequate for the job as long as it's usually up.

OK, so you're running Windows, eh? (I am assuming here that you are using an actual OS like 2000 or XP. If you're using Windows 98 or God help you, 95 or ME, there may be a lot of extra steps, and I will provide zero support.) Head here and download the latest svn-<version number>-setup.exe. (Don't pick anything with "rc" in it, that means "Release Candidate", i.e., not quite ready.) As of this writing the latest release is 1.0.6, but they come out with new versions typically once or twice a month. So, download the setup.exe and install it (it won't ask for anything out of the ordinary for installers).

Setting Up Your Computer

The first step is to make sure that you have your computer correctly set up. Oftentimes mod teams become very frustrated since everyone uses different directories for their mods and code, ensuring that the projects are always messed up (this has become even worse since Steam puts your email address right in the directory). However, this is easily solved by using the Windows command "subst". (I'm pretty sure this doesn't exist in Windows 98, let me know if it does.) Subst takes a directory and maps it to a drive letter. Let's assume that your source code's folder is rooted at C:\projects\mymod\source, and your mod's directory is at C:\steam\steamapps\myemailaddress\mymod. Then all you need do is open up a command prompt (if you're on 2000, make sure you run "cmd", not "command", which is a deprecated DOS interpreter but looks similar), and type the following:

 CODE  
subst S: C:\projects\mymod\source
subst M: C:\steam\steamapps\myemailaddress\mymod


Now you will find if you go to S:, that your source code is in there, and if you go to M:, your mod is there. Note that these are just virtual drives, so changes you make to files on these drives will also be reflected on the C drive. Anyway, though, this should make things a lot easier for you in terms of project files and where to copy DLLs and so forth by standardizing file locations across computers. Instead of linking DLLs to C:\steam\steamapps\etc\etc you simply link them to M:\dlls and M:\cl_dll. Also, it makes navigating in Windows Explorer a little easier when you're not three or four directories deep into a drive. I will be assuming you are using the letters S: and M:, but of course, on your own project you may use whatever you wish. Generally you'll want to make the two commands above into a batch file and put that into your Startup folder in the Start menu, as these drives do not persist over reboots.

Setting Up The Repository

Now you'll need to actually set up your repository. These instructions are all directly copied out of the readme.txt and various other help files in the documentation, so you actually might be better off going through it yourself, you'll understand it a little better. Their readme.txt is a nice quick get-you-started, while their documentation help file is very long and detailed, quite impressive documentation really.

First you need to choose a directory to keep the repository files in. Note that this is VERY DIFFERENT from where the files actually reside, i.e., S:. The repository files are the SERVER files -- the actual files you have on your computer will be copies of the client files. I'm going to assume that you keep them in C:\trunk (you don't have to create it). Go to the directory where you installed Subversion, and go to the "bin" directory (in your command prompt). Type in the following:

 CODE  
svnadmin create C:\trunk


There you go, your repository is created. Now, don't go looking for source code files in there -- these are mostly kept in BerkeleyDB databases, nothing in C:\trunk is human-readable. However, at present, our SCM doesn't have any files actually in it. We need to add some. Type the following in (same directory as before):

 CODE  
svn import S:\ file:///trunk -m "Initial Import"


This will import all the files under S:\ into the repository, with the commit message "Initial Import". Note that above I put file:///trunk, which has no specified drive letter. For that reason, you may need to create your repository on the same drive that you have Subversion installed on. You should get the following (I have two files, sc1.cpp and sc2.cpp. You should see many more files.)

 CODE  
Adding         S:\sc1.cpp
Adding         S:\sc2.cpp

Committed revision 1.


OK, now, move all the files and directories that were in S:\ to some other location. The files are now stored in the repository, so you could even delete them. Don't do that until you do the next step and make sure it works, though. Type:

 CODE  
svn update S:\


This will update your files with the files that were in the repository. You should get the following:

 CODE  
Restored 'S:\sc1.cpp'
Restored 'S:\sc2.cpp'
At revision 1.


If it said all that, you've got your repository up and working! Give yourself a pat on the back!

Making The Repository Available On The Internet

Now, note that it isn't yet actually available on the Internet yet, only on your local computer. We have to set up svnserve.exe. Now, I want to go ahead and warn you that svnserve.exe may not be the best thing here, because it may have a weird and possibly broken implementation under Windows. I don't know if they've fixed this yet. Nonetheless, that's life. It'll do commits, updates, and checkouts. I would suggest that once your coders get familiar with SVN and you really feel like you want to keep on using it, or if the following is just WAY TOO ANNOYING for you, then TortoiseSVN has some really nice docs on getting the whole Apache-Subversion thing set up in Windows. They also have an explanation of SVNserve.exe but they're going to tell you basically what I'm going to tell you.

What you're going to want to do is create a batch file that you'll put in the Startup folder. (Since you've already got your subst commands in there, just add this at the end.) Put the following in there:

 CODE  
C:\yoursubversionfolder\bin\svnserve.exe -d --listen-port yourport -r C:\trunk


Of course, replace yoursubversionfolder with the folder you installed Subversion into and yourport with whatever port you want your server to listen on (default is 3960 if you don't use the listen-port option). Now, when you restart your computer, you should see a command prompt come up, run the subst commands, run svnserve, and then stay open. Yup, the brilliant minds at Windows made it about as hard as possible to run something in the background. Thus, you have three options. One is to live with it. Minimize the window, leave it open, don't accidentally close it. The second option is to install svnserve.exe as a service, using firedaemon or something similar. The other is to start downloading Apache. If you do that, though, it shouldn't change any of what I'm about to tell you, so just do that for now and step it up later.

By the way, the "-r C:\trunk" option above sets the trunk directory as the root of your repository, so that later you can type svn://yourserver/ instead of svn://yourserver/trunk. Just a little helpful thing.

Now, you'll also need to configure svnserve.exe correctly. Go to C:\trunk\conf and open up svnserve.conf. Basically, delete everything and put the following:

 CODE  
[general]
anon-access = none
auth-access = write
password-db = password.conf


This will make it so that no one can get your files but your authorized users. If you set anon-access to read, then anyone who wants to can download your files and look at them, but they can't write to them.

You'll now need to open up a file called password.conf, again in C:\trunk\conf. You'll want to put the following:

 CODE  
[users]
user1 = pass1
user2 = pass2
user3 = pass3


(Where of course user* and pass* are replaced by appropriate names and passwords for your developers.) If you haven't already figured it out, you're probably going to want your coders to use something other than their secure password that they use for everything (we all do it :D), because their passwords are going to be in plaintext on your computer. Also, all the coders could use the same user name, but you don't want to do that because you want to be able to see who did which change. Now, your server should be all set up. Let's go to the client side!

OK, I've Got A Repository. But Using The Command Line Sucks. What's Next?

Setting Up TortoiseSVN

Now we would like to set up TortoiseSVN on the client side. Off you go to the TortoiseSVN download page. Again, don't pick Release Candidates or special versions, just pick the latest, which happens to be 1.0.8.1430 at this writing. These come in MSI format, which is an automatic Windows installer. Again, the installation will present no difficulties, although be aware that it will restart your computer. Also, be aware that this is a shell extension, so it will put another couple of items in your right-click folder. Having it incorporated into the shell is a very nice feature, though.

Open up Windows Explorer, and go to S:. (If it's not there, remember that you'll either have to redo the subst commands every time you reboot or put it in a batch file and put it in your Startup folder.) It should be filled with icons with little green check marks on them. However, other coders who don't have the files yet (they cannot use their old SDK copies, they must get a new copy from the Subversion folder so make sure there aren't any extensive changes to the code base outstanding) should read the next paragraph and follow what it says to get them. But you won't need to do that, as you already have the files.

Right-click anywhere in the directory, go to TortoiseSVN and then to Checkout. It will ask you for the URL of the repository. You should put in svn://yourserverIPhere:yourserverporthere/. That's it. If the SVN repository is on your computer, you can use svn://localhost:yourport/ or simply file:///trunk. The TortoiseSVN program should download all the files you need into the folder. You'll notice they all have little green checkmarks on them. That means that they're up-to-date.

Getting To Know TortoiseSVN

OK, so now we're going to explore how to use TortoiseSVN a bit. Open up a file, any file, make a small change, and save. In Windows Explorer, you'll notice the little green mark has changed to a red exclamation point. That's how you know the files are changed. Right-click on it, go to TortoiseSVN, and hit "Revert". Your changes are instantly blasted away, and the file is as it was before, with a green check mark. Open up the file again, change it, and save. Now, right-click and hit "Commit". This will bring up a dialog box with the file you changed and a text box where you can add a commit comment. Try double-clicking on the file name. You should get a new window which will show the old and the new versions on the left and right side. Note how it says "TortoiseMerge" at the top left. This is the program you will use to merge changes later, but for right now it only shows the differences. Close TortoiseMerge and go back to the dialog box. Move your mouse over the "files total" on the bottom right and you'll see a helpful information bubble summarizing the states your files are in. Note that if you have a conflicted file in there, you SHOULD NOT BE COMMITTING. Finally, put in a commit comment and hit Commit. It will ask for your username and password (hit Save Authentication if you don't want to put them in every time). Type them in, hit OK, and boom, your file should be right back to green check mark land.

This will cause your files to be committed to the repository, and now anyone who updates will get your latest file. Try it -- delete the file and then right-click in the folder and click "Update". Your changed file should come right back up. Right-click on the file, go to TortoiseSVN, and click "Update to Revision". Put in "1" and hit Update. Your file will now be its old self again, although on the repository the latest file is still considered to be your new one. (Note, by the way, that you don't have to select the files you want to commit. Commit from the root folder of your project. TortoiseSVN will check what files have been changed and show them all for your approval.)

Merging Conflicted Files

Now, what happens if two people make mutually exclusive changes to files? Well, we'll have to use the merge tool mentioned earlier. I didn't mention it earlier, but you need to ALWAYS Update at the root folder before you make commitments. This will check the server and see if there are any conflicts. If there are, it will tell you about them, and create "sc2.cpp.mine", "sc2.cpp.r5", and "sc2.cpp.r6", where sc2.cpp is the file, r5 and r6 are the last version and the conflicting version, and mine holds the file you created. You'll note that your file has a big triangle with an exclamation point. Right-click on the file, go to TortoiseSVN, and use "Edit Conflicts".

TortoiseMerge will pop up on the screen again, with "Theirs" and "Yours" on the left and right side, and the merged file at the bottom. It'll have a bunch of lines saying "Unresolved Conflict!!!" Basically what you do here is go through the files, right clicking on the lines and selecting the code from "Theirs" or "Yours". This is sort of a pain but resolving code conflicts is. As long as two coders aren't actually working on the same thing at the same time, and you're committing your changes in a reasonably timely manner, this probably will not be much of a problem. Most conflicts are very easily resolved by SVN because they aren't to the same lines of code. Once all the Unresolved Conflict lines are gone, you hit Save and then exit Tortoise Merge. Now right click on the file again, go to TortoiseSVN, and click "Resolved". The triangle sign changes to a regular exclamation point, and now you may right-click and hit Commit. And there you go, it's just that easy.

Adding And Deleting Files

One more thing you need to know -- how to add files to a repository. Simply create a new file, right-click, go to TortoiseSVN, and hit Add. Then Commit the file. Deleting files from a repository is just as easy -- just delete the actual working file and then Commit the folder.

That's basically going to cover the vast majority of your life. Whenever you start to work, make it a habit to right-click in the root folder of your project and click "Update". This will bring all your files up-to-date and let you know of any conflicts you need to resolve. Once you're ready to commit your files, Update first and then if there's no problem, Commit them. One last thing to note -- right-click on the file you've been messing with, go to TortoiseSVN, and click "Show Log". This will show a log of all the changes that have been made to that file. You can check the differences between any one revision and your working file by double-clicking on the revision, or any two old revisions by selecting both, right-clicking and choosing to compare them.

Hey, SCM Is Really Cool!

I know, right? There will be some growing pains at first -- people grumbling about problems they're having because they're just not used to it. But SCM is ultimately very helpful and you'll find that once your guys get used to it, they'll love it. Also, keep in mind, screwing something up in an SCM is just about the easiest thing in the world to fix -- just back up the revision and recommit. Get them to read the TortoiseSVN documents -- they're really pretty good. Once you get used to it, you'll find editing code and keeping everything up-to-date so easy you'll wonder how you ever got along without it.

Subversion doesn't just keep text files, though, it also keeps binary files, so your mappers, modelers, and texture artists can upload their source files and their final completed files into the repository as well. Having more than one repository on a computer is very easy. I would suggest having one repository for art source files, one repository for code source files, and one repository for your mod object data if you really want to get into it. Of course, binary data can't be diffed or merged, but Subversion will still keep track of all the revisions to all your old files in the exact same way that it did your code files. Again, this is a little less necessary because artists typically don't work on the same thing at the same time. It is a really nice way to keep everyone's files up to date. Note also that you can allow beta testers read-only access to the mod object file repository, so that they can easily and quickly download the very latest beta every single day.

Anyway, I appreciate your taking the time to read my humble article, and I hope the mod community will be able to become a little more organized in how they handle their source code. The HL project is a big thing, with hundreds of files, and yet many mod teams stumble along using IRC, FTP and Winzip to handle things, even while crappy little Sourceforge projects are using CVS to control their ten files. It's ridiculous. Sure, the making of a mod is an amateur project, but it doesn't have to be done amateurishly while there are completely free tools to help you out. SCM is ESPECIALLY helpful for mod teams vis-a-vis commercial teams because there's typically very little communication compared to a commercial team all working in the same building.

Thanks for reading.

Rate This Article
This article is currently rated: 4.2 out of 5.0 (6 Votes)

You have to register to rate this article.
User Comments Showing comments 1-10

Posted By: IoN_PuLse on Aug 22 2004 at 08:11:20
Here here to using subversion!!

Posted By: DarkNight on Aug 22 2004 at 08:22:21
You go girl!

I wonder if anyone will use this great system...

Posted By: deathz0rz on Aug 24 2004 at 11:10:47
note that there is also something like cvs that does almost the same

Posted By: Persuter on Aug 24 2004 at 13:29:21
CVS is mentioned in the article, actually.

Posted By: Put Put Cars on Aug 24 2004 at 21:21:24
SOunds Cool. Improved CVS I guess.

Posted By: chbrules on Aug 29 2004 at 09:57:18
I'm going to try it, never used it before, but the theory is great!

Posted By: chbrules on Aug 29 2004 at 12:20:53
Ok, I don't know if I'm alone, but I got confused. You should focus more on the details of to do what client side and what server side. I thought you meant create the virtual drive on the server side until I figured it out on client side! Pretty good overall though, my first CVS experience! =D

Posted By: Vino on Oct 22 2004 at 14:04:22
I can't believe you Persuter, you totally forgot to mention the BOOK!

http://svnbook.red-bean.com/svnbook-1.0/

It's a free book on how to use SVN to do all of the things he's talking about, and it's great reference material.

Posted By: NuclearFriend on Oct 22 2004 at 17:43:47
404'd! :)

http://svnbook.red-bean.com/svnbook-1.0/svn-book.html
http://svnbook.red-bean.com/svnbook-1.1/svn-book.htmlEdited by NuclearFriend on Oct 22 2004, 17:44:08

Posted By: Vino on Oct 22 2004 at 22:01:01
Oh noes! Sorry about that. Nuke is right, hit up http://svnbook.red-bean.com/svnbook-1.1/svn-book.html


You must register to post a comment. If you have already registered, you must login.

Latest Articles
3rd person View in Multiplayer
Half-Life 2 | Coding | Client Side Tutorials
How to enable it in HL2DM

By: cct | Nov 13 2006

Making a Camera
Half-Life 2 | Level Design
This camera is good for when you join a map, it gives you a view of the map before you join a team

By: slackiller | Mar 05 2006

Making a camera , Part 2
Half-Life 2 | Level Design
these cameras are working monitors that turn on when a button is pushed.

By: slackiller | Mar 04 2006

Storing weapons on ladder
Half-Life 2 | Coding | Snippets
like Raven Sheild or BF2

By: British_Bomber | Dec 24 2005

Implementation of a string lookup table
Half-Life 2 | Coding | Snippets
A string lookup table is a set of functions that is used to convert strings to pre-defined values

By: deathz0rz | Nov 13 2005


Latest Comments
knock knock
General | News
By: omega | Dec 22 2016
 
knock knock
General | News
By: MIFUNE | Oct 10 2015
 
New HL HUD Message System
Half-Life | Coding | Shared Tutorials
By: chbrules | Dec 31 2011
 
knock knock
General | News
By: Whistler | Nov 05 2011
 
Particle Engine tutorial part 4
Half-Life | Coding | Client Side Tutorials
By: darkPhoenix | Feb 18 2010
 
Particle Engine tutorial part 2
Half-Life | Coding | Client Side Tutorials
By: darkPhoenix | Feb 11 2010
 
Particle Engine tutorial part 3
Half-Life | Coding | Client Side Tutorials
By: darkPhoenix | Feb 11 2010
 
Game Movement Series #2: Analog Jumping and Floating
Half-Life 2 | Coding | Shared Tutorials
By: mars3554 | Oct 26 2009
 
Particle Engine tutorial part 5
Half-Life | Coding | Client Side Tutorials
By: Deadpool | Aug 02 2009
 
Particle Engine tutorial part 5
Half-Life | Coding | Client Side Tutorials
By: Persuter | Aug 02 2009
 

Site Info
297 Approved Articless
8 Pending Articles
3940 Registered Members
0 People Online (8 guests)
About - Credits - Contact Us

Wavelength version: 3.0.0.9
Valid XHTML 1.0! Valid CSS!