3 Version Control System Operations

It is assumed that you are already familiar with the basic operation of the version control systems in use. Traditionally this was CVS, but as of June 2008, Subversion is used for the src tree.

The CVS Repository Meisters <cvsadm@FreeBSD.org> are the “owners” of the repository and are responsible for direct modification of it for the purposes of cleanup or fixing some unfortunate abuse of the version control system by a committer. Should you cause some repository accident, say a bad import or a bad tag creation, mail the responsible part of CVS Repository Meisters <cvsadm@FreeBSD.org>, as stated in the table below, (or call one of them) and report the problem. For very important issues affecting the entire tree--not just a specific area--you can contact the CVS Repository Meisters <cvsadm@FreeBSD.org>. Please do not contact the CVS Repository Meisters <cvsadm@FreeBSD.org> for repocopies or other things that the more specific teams can handle.

The only ones able to directly fiddle the repository bits on the repository hosts are the repomeisters. To enforce this, there are no login shells available on the repository machines, except to the repomeisters.

The FreeBSD repositories are currently split into four distinct parts, namely doc, ports, projects and src. These are combined under a single CVSROOT when distributed via CVSup for the convenience of our users. The src tree is automatically exported to CVS for compatibility reasons only (e.g. CVSup). The “official” src repository is not stored in CVS but in Subversion. The official and exported trees are not necessarily equal.

The CVS repositories are hosted on the repository machines. Currently, each of the repositories above reside on the same physical machine, ncvs.FreeBSD.org, but to allow for the possibility of placing each on a separate machine in the future, there is a separate hostname for each that committers should use. Additionally, each repository is stored in a separate directory. The following table summarizes the situation.

CVS operations are done remotely by setting the CVSROOT environment variable to the appropriate host and top-level directory (for example, dcvs.FreeBSD.org:/home/dcvs), and doing the appropriate check-out/check-in operations. Many committers define aliases which expand to the correct cvs invocation for the appropriate repository. For example, a tcsh(1) user may add the following to their .cshrc for this purpose:

alias dcvs cvs -d user@dcvs.FreeBSD.org:/home/dcvs
alias pcvs cvs -d user@pcvs.FreeBSD.org:/home/pcvs
alias projcvs cvs -d user@projcvs.FreeBSD.org:/home/projcvs

This way they can do all CVS operations locally and use Xcvs commit for committing to the official CVS repository. Refer to the cvs(1) manual page for usage.

If you need to use CVS add and delete operations in a manner that is effectively a mv(1) operation, then a repository copy is in order rather than using CVS add and delete. In a repository copy, a repomeister will copy the file(s) to their new name and/or location and let you know when it is done. The purpose of a repository copy is to preserve file change history, or logs. We in the FreeBSD Project greatly value the change history that a version control system gives to the project.

CVS reference information, tutorials, and FAQs can be found at: http://www.cvshome.org/docs/. The information in Karl Fogel's chapters from “Open Source Development with CVS” is also very useful.

Dag-Erling C. Smørgrav <des@FreeBSD.org> also supplied the following “mini primer” for CVS.

1. Check out a module with the co or checkout command.

   % cvs checkout shazam
   

   This checks out a copy of the shazam module. If there is no shazam module in the modules file, it looks for a top-level directory named shazam instead.

   Table 2. Useful cvs checkout options

   -P	Do not create empty directories
   -l	Check out a single level, no subdirectories
   -rrev	Check out revision, branch or tag rev
   -Ddate	Check out the sources as they were on date date

   Practical FreeBSD examples:

   • Check out the Tools module, which corresponds to ports/Tools:

     % cvs co Tools
     

     You now have a directory named ports/Tools with subdirectories portbuild, scripts, and CVS.

   • Check out the same files, but with full path:

     % cvs co ports/Tools
     

     You now have a directory named ports, with subdirectories CVS and Tools. The ports/Tools directory has subdirectories CVS and scripts, etc.

   • Check out the directory Tools, but none of the subdirectories:

     % cvs co -l Tools
     

     You now have a directory named Tools with just one subdirectory named CVS.

   • Check out the Tools module as it was when support for FreeBSD 5.X was dropped:

     % cvs co -rRELEASE_5_EOL Tools
     

     You will not be able to commit modifications, since RELEASE_5_EOL is a point in time, not a branch.

   • Check out the Tools module as it was on March 25th, 2009:

     % cvs co -D'2009-03-25' Tools
     

     You will not be able to commit modifications.

   • Check out the Tools module as it was one week ago:

     % cvs co -D'last week' Tools
     

     You will not be able to commit modifications.

   Note that cvs stores metadata in subdirectories named CVS. Similarly, Subversion stores metadata in subdirectories named .svn.

   Arguments to -D and -r are sticky, which means cvs will remember them later, e.g. when you do a cvs update.

2. Check the status of checked-out files with the status command.

   % cvs status shazam
   

   This displays the status of the file shazam or of every file in the shazam directory. For every file, the status is given as one of:

   Up-to-date	File is up-to-date and unmodified.
   Needs Patch	File is unmodified, but there is a newer revision in the repository.
   Locally Modified	File is up-to-date, but modified.
   Needs Merge	File is modified, and there is a newer revision in the repository.
   File had conflicts on merge	There were conflicts the last time this file was updated, and they have not been resolved yet.

   You will also see the local revision and date, the revision number of the newest applicable version (“newest applicable” because if you have a sticky date, tag or branch, it may not be the actual newest revision), and any sticky tags, dates or options.

3. Once you have checked something out, you can update it with the update command.

   % cvs update shazam
   

   This updates the file shazam or the contents of the shazam directory to the latest version along the branch you checked out. If you checked out a “point in time”, it does nothing unless the tags have moved in the repository or some other weird stuff is going on.

   Useful options, in addition to those listed above for checkout:

   -d	Check out any additional missing directories.
   -A	Update to head of main branch.

   If you checked out a module with -r or -D, running cvs update with a different -r or -D argument or with -A will select a new branch, revision or date. The -A option clears all sticky tags, dates or revisions whereas -r and -D set new ones.

   Theoretically, specifying HEAD as the argument to -r will give you the same result as -A, but that is just theory.

   The -d option is useful if:

   • somebody has added subdirectories to the module you have checked out after you checked it out.

   • you checked out with -l, and later change your mind and want to check out the subdirectories as well.

   • you deleted some subdirectories and want to check them all back out.

   Watch the output of the cvs update with care. The letter in front of each filename indicates what was done with it:

   U	The file was updated without trouble.
   P	The file was updated without trouble (you will only see this when working against a remote repository).
   M	The file had been modified, and was merged without conflicts.
   C	The file had been modified, and was merged with conflicts.

   Merging is what happens if you check out a copy of some file, modify it, then someone else commits a change, and you run cvs update. CVS notices that you have made local changes, and tries to merge your changes with the changes between the version you originally checked out and the one you updated to. If the changes are to separate portions of the file, it will almost always work fine (though the result might not be syntactically or semantically correct).

   CVS will print an M in front of every locally modified file even if there is no newer version in the repository, so cvs update is handy for getting a summary of what you have changed locally.

   If you get a C, then your changes conflicted with the changes in the repository (the changes were to the same lines, or neighboring lines, or you changed the local file so much that cvs can not figure out how to apply the repository's changes). You will have to go through the file manually and resolve the conflicts; they will be marked with rows of <, = and > signs. For every conflict, there will be a marker line with seven < signs and the name of the file, followed by a chunk of what your local file contained, followed by a separator line with seven = signs, followed by the corresponding chunk in the repository version, followed by a marker line with seven > signs and the revision number you updated to.

4. View differences between the local version and the repository version with the diff command.

   % cvs diff shazam
   

   shows you every modification you have made to the shazam file or module.

   Table 3. Useful cvs diff options

   -u	Uses the unified diff format.
   -c	Uses the context diff format.
   -N	Shows missing or added files.

   You always want to use -u, since unified diffs are much easier to read than almost any other diff format (in some circumstances, context diffs generated with the -c option may be better, but they are much bulkier). A unified diff consists of a series of hunks. Each hunk begins with a line that starts with two @ signs and specifies where in the file the differences are and how many lines they span. This is followed by a number of lines; some (preceded by a blank) are context; some (preceded by a - sign) are outtakes and some (preceded by a +) are additions.

   You can also diff against a different version than the one you checked out by specifying a version with -r or -D as in checkout or update, or even view the diffs between two arbitrary versions (without regard for what you have locally) by specifying two versions with -r or -D.

5. View log entries with the log command.

   % cvs log shazam
   

   If shazam is a file, this will print a header with information about this file, such as where in the repository this file is stored, which revision is the HEAD for this file, what branches this file is in, and any tags that are valid for this file. Then, for each revision of this file, a log message is printed. This includes the date and time of the commit, who did the commit, how many lines were added and/or deleted, and finally the log message that the committer who did the change wrote.

   If shazam is a directory, then the log information described above is printed for each file in the directory in turn. Unless you give the -l to log, the log for all subdirectories of shazam is printed too, in a recursive manner.

   Use the log command to view the history of one or more files, as it is stored in the CVS repository. You can even use it to view the log message of a specific revision, if you add the -rrev to the log command:

   % cvs log -r1.2 shazam
   

   This will print only the log message for revision 1.2 of file shazam if it is a file, or the log message for revision 1.2 of each file under shazam if it is a directory.

6. See who did what with the annotate command. This command shows you each line of the specified file or files, along with which user most recently changed that line.

   % cvs annotate shazam
   

7. Add new files with the add command.

   Create the file, cvs add it, then cvs commit it.

   Similarly, you can add new directories by creating them and then cvs adding them. Note that you do not need to commit directories.

8. Remove obsolete files with the remove command.

   Remove the file, then cvs rm it, then cvs commit it.

9. Commit with the commit or checkin command.

   Table 4. Useful cvs commit options

   -f	Force a commit of an unmodified file.
   -mmsg	Specify a commit message on the command line rather than invoking an editor.

   The following are some Subversion examples related to the src repository. More (in-depth) information can be found at Subversion Primer and List of things missing in Subversion when compared to CVS. The notes at http://people.freebsd.org/~peter/svn_notes.txt might also be useful. Subversion is also described in-depth in Version Control with Subversion.

   • Check out the head branch:

     % svn co svn+ssh://svn.freebsd.org/base/head /usr/src
     

   Good commit messages are important. They tell others why you did the changes you did, not just right here and now, but months or years from now when someone wonders why some seemingly illogical or inefficient piece of code sneaked into your source file. It is also an invaluable aid to deciding which changes to MFC and which not to MFC.

   Commit messages should be clear, concise and provide a reasonable summary to give an indication of what was changed and why.

   Commit messages should provide enough information to enable a third party to decide if the change is relevant to them and if they need to read the change itself.

   Avoid committing several unrelated changes in one go. It makes merging difficult, and also makes it harder to determine which change is the culprit if a bug crops up.

   Avoid committing style or whitespace fixes and functionality fixes in one go. It makes merging difficult, and also makes it harder to understand just what functional changes were made. In the case of documentation files, it can make the job of the translation teams more complicated, as it becomes difficult for them to determine exactly what content changes need to be translated.

   Avoid committing changes to multiple files in one go with a generic, vague message. Instead, commit each file (or small, related groups of files) with tailored commit messages.

   Before committing, always:

   • verify which branch you are committing to, using svn status. This is only needed for the src tree, as the other trees are not branched.

   • review your diffs, using the diff command of the version control system.

   Also, ALWAYS specify which files to commit explicitly on the command line, so you do not accidentally commit other files than the ones you intended -- a commit operation without any arguments usually will commit every modification in your current working directory and every subdirectory.

Additional tips and tricks:

1. You can place commonly used options in your ~/.cvsrc, like this:

   cvs -z3
   diff -Nu
   update -Pd
   checkout -P
   

   This example says:

   • always use compression level 3 when talking to a remote server. This is a life-saver when working over a slow connection.

   • always use the -N (show added or removed files) and -u (unified diff format) options to diff(1).

   • always use the -P (prune empty directories) and -d (check out new directories) options when updating.

   • always use the -P (prune empty directories) option when checking out.

2. Use Eivind Eklund's cdiff script to view unidiffs. It is a wrapper for less(1) that adds ANSI color codes to make hunk headers, outtakes and additions stand out; context and garbage are unmodified. It also expands tabs properly (tabs often look wrong in diffs because of the extra character in front of each line).

   textproc/cdiff

   Simply use it instead of more(1) or less(1):

   % cvs diff -Nu shazam | cdiff
   

   Alternatively some editors like vim(1) (editors/vim) have color support and when used as a pager with color syntax highlighting switched on will highlight many types of file, including diffs, patches, and CVS/RCS logs.

   % echo "syn on" >> ~/.vimrc 
   % cvs diff -Nu shazam | vim -
   % cvs log shazam | vim -
   

3. CVS is old, arcane, crufty and buggy, and sometimes exhibits non-deterministic behavior which some claim as proof that it is actually merely the Newtonian manifestation of a sentient transdimensional entity. It is not humanly possible to know its every quirk inside out, so do not be afraid to ask the resident AI (CVS Repository Meisters <cvsadm@FreeBSD.org>) for help.

4. Do not leave the cvs commit command in commit message editing mode for too long (more than 2-3 minutes). It locks the directory you are working with and will prevent other developers from committing into the same directory. If you have to type a long commit message, type it before executing cvs commit and insert it into the commit message or save it in a file before committing and use the -F option of CVS to read the commit message from that file, i.e.

   % vi logmsg
   % cvs ci -F logmsg shazam
   

   This is the fastest way of passing a commit message to CVS but you should be careful when editing the logmsg file before the commit, because CVS will not give you a chance to edit the message when you do the actual commit.

5. Speed up your CVS operation considerably by using a persistent ssh connection to the repository machine. First, put this configuration into your ~/.ssh/config:

   Host dcvs.FreeBSD.org
         ControlPath /home/user/.ssh/cvs.cpath
   Host projcvs.FreeBSD.org
         ControlPath /home/user/.ssh/cvs.cpath
   Host pcvs.FreeBSD.org
         ControlPath /home/user/.ssh/cvs.cpath
   

   Now open the persistent connection to the repoman:

   % ssh -fNM ncvs.FreeBSD.org
   

   The CVS commands should now respond faster, as they are reusing existing connection with the repository. Note that all the hostnames are case sensitive.