A.6 Using CVSup

A.6.1 Introduction

CVSup is a software package for distributing and updating source trees from a master CVS repository on a remote server host. The FreeBSD sources are maintained in a CVS repository on a central development machine in California. With CVSup, FreeBSD users can easily keep their own source trees up to date.

CVSup uses the so-called pull model of updating. Under the pull model, each client asks the server for updates, if and when they are wanted. The server waits passively for update requests from its clients. Thus all updates are instigated by the client. The server never sends unsolicited updates. Users must either run the CVSup client manually to get an update, or they must set up a cron job to run it automatically on a regular basis.

The term CVSup, capitalized just so, refers to the entire software package. Its main components are the client cvsup which runs on each user's machine, and the server cvsupd which runs at each of the FreeBSD mirror sites.

As you read the FreeBSD documentation and mailing lists, you may see references to sup. Sup was the predecessor of CVSup, and it served a similar purpose. CVSup is used much in the same way as sup and, in fact, uses configuration files which are backward-compatible with sup's. Sup is no longer used in the FreeBSD project, because CVSup is both faster and more flexible.

Note: The csup utility is a rewrite of the CVSup software in C. Its biggest advantage is, that it is faster and does not depend on the Modula-3 language, thus you do not need to install it as a requirement. Moreover, if you are using FreeBSD 6.2 or later, you can use it out-of-the-box, since it is included in the base system. Older FreeBSD versions do not have csup(1) in their base system but you can easily install the net/csup port, or a precompiled package. The csup utility does not support CVS mode, though. If you want to mirror complete repositories, you will still need to use CVSup. If you decided to use csup, just skip the steps on the installation of CVSup and substitute the references of CVSup with csup while following the remainder of this article.

A.6.2 Installation

The easiest way to install CVSup is to use the precompiled net/cvsup package from the FreeBSD packages collection. If you prefer to build CVSup from source, you can use the net/cvsup port instead. But be forewarned: the net/cvsup port depends on the Modula-3 system, which takes a substantial amount of time and disk space to download and build.

Note: If you are going to be using CVSup on a machine which will not have XFree86 or Xorg installed, such as a server, be sure to use the port which does not include the CVSup GUI, net/cvsup-without-gui.

If you want to install csup on FreeBSD 6.1 or earlier, you can use the precompiled net/csup package from the FreeBSD packages collection. If you prefer to build csup from source, you can use the net/csup port instead.

A.6.3 CVSup Configuration

CVSup's operation is controlled by a configuration file called the supfile. There are some sample supfiles in the directory /usr/share/examples/cvsup/.

The information in a supfile answers the following questions for CVSup:

In the following sections, we will construct a typical supfile by answering each of these questions in turn. First, we describe the overall structure of a supfile.

A supfile is a text file. Comments begin with # and extend to the end of the line. Lines that are blank and lines that contain only comments are ignored.

Each remaining line describes a set of files that the user wishes to receive. The line begins with the name of a “collection”, a logical grouping of files defined by the server. The name of the collection tells the server which files you want. After the collection name come zero or more fields, separated by white space. These fields answer the questions listed above. There are two types of fields: flag fields and value fields. A flag field consists of a keyword standing alone, e.g., delete or compress. A value field also begins with a keyword, but the keyword is followed without intervening white space by = and a second word. For example, release=cvs is a value field.

A supfile typically specifies more than one collection to receive. One way to structure a supfile is to specify all of the relevant fields explicitly for each collection. However, that tends to make the supfile lines quite long, and it is inconvenient because most fields are the same for all of the collections in a supfile. CVSup provides a defaulting mechanism to avoid these problems. Lines beginning with the special pseudo-collection name *default can be used to set flags and values which will be used as defaults for the subsequent collections in the supfile. A default value can be overridden for an individual collection, by specifying a different value with the collection itself. Defaults can also be changed or augmented in mid-supfile by additional *default lines.

With this background, we will now proceed to construct a supfile for receiving and updating the main source tree of FreeBSD-CURRENT.

A.6.3.1 The refuse File

As mentioned above, CVSup uses a pull method. Basically, this means that you connect to the CVSup server, and it says, “Here is what you can download from me...”, and your client responds “OK, I will take this, this, this, and this.” In the default configuration, the CVSup client will take every file associated with the collection and tag you chose in the configuration file. However, this is not always what you want, especially if you are synching the doc, ports, or www trees -- most people cannot read four or five languages, and therefore they do not need to download the language-specific files. If you are CVSuping the Ports Collection, you can get around this by specifying each collection individually (e.g., ports-astrology, ports-biology, etc instead of simply saying ports-all). However, since the doc and www trees do not have language-specific collections, you must use one of CVSup's many nifty features: the refuse file.

The refuse file essentially tells CVSup that it should not take every single file from a collection; in other words, it tells the client to refuse certain files from the server. The refuse file can be found (or, if you do not yet have one, should be placed) in base/sup/. base is defined in your supfile; our defined base is /var/db, which means that by default the refuse file is /var/db/sup/refuse.

The refuse file has a very simple format; it simply contains the names of files or directories that you do not wish to download. For example, if you cannot speak any languages other than English and some German, and you do not feel the need to read the German translation of documentation, you can put the following in your refuse file:

doc/bn_*
doc/da_*
doc/de_*
doc/el_*
doc/es_*
doc/fr_*
doc/hu_*
doc/it_*
doc/ja_*
doc/mn_*
doc/nl_*
doc/no_*
doc/pl_*
doc/pt_*
doc/ru_*
doc/sr_*
doc/tr_*
doc/zh_*

and so forth for the other languages (you can find the full list by browsing the FreeBSD CVS repository).

With this very useful feature, those users who are on slow links or pay by the minute for their Internet connection will be able to save valuable time as they will no longer need to download files that they will never use. For more information on refuse files and other neat features of CVSup, please view its manual page.

A.6.4 Running CVSup

You are now ready to try an update. The command line for doing this is quite simple:

# cvsup supfile

where supfile is of course the name of the supfile you have just created. Assuming you are running under X11, cvsup will display a GUI window with some buttons to do the usual things. Press the go button, and watch it run.

Since you are updating your actual /usr/src tree in this example, you will need to run the program as root so that cvsup has the permissions it needs to update your files. Having just created your configuration file, and having never used this program before, that might understandably make you nervous. There is an easy way to do a trial run without touching your precious files. Just create an empty directory somewhere convenient, and name it as an extra argument on the command line:

# mkdir /var/tmp/dest
# cvsup supfile /var/tmp/dest

The directory you specify will be used as the destination directory for all file updates. CVSup will examine your usual files in /usr/src, but it will not modify or delete any of them. Any file updates will instead land in /var/tmp/dest/usr/src. CVSup will also leave its base directory status files untouched when run this way. The new versions of those files will be written into the specified directory. As long as you have read access to /usr/src, you do not even need to be root to perform this kind of trial run.

If you are not running X11 or if you just do not like GUIs, you should add a couple of options to the command line when you run cvsup:

# cvsup -g -L 2 supfile

The -g tells CVSup not to use its GUI. This is automatic if you are not running X11, but otherwise you have to specify it.

The -L 2 tells CVSup to print out the details of all the file updates it is doing. There are three levels of verbosity, from -L 0 to -L 2. The default is 0, which means total silence except for error messages.

There are plenty of other options available. For a brief list of them, type cvsup -H. For more detailed descriptions, see the manual page.

Once you are satisfied with the way updates are working, you can arrange for regular runs of CVSup using cron(8). Obviously, you should not let CVSup use its GUI when running it from cron(8).

A.6.5 CVSup File Collections

The file collections available via CVSup are organized hierarchically. There are a few large collections, and they are divided into smaller sub-collections. Receiving a large collection is equivalent to receiving each of its sub-collections. The hierarchical relationships among collections are reflected by the use of indentation in the list below.

The most commonly used collections are src-all, and ports-all. The other collections are used only by small groups of people for specialized purposes, and some mirror sites may not carry all of them.

cvs-all release=cvs

The main FreeBSD CVS repository, including the cryptography code.

distrib release=cvs

Files related to the distribution and mirroring of FreeBSD.

doc-all release=cvs

Sources for the FreeBSD Handbook and other documentation. This does not include files for the FreeBSD web site.

ports-all release=cvs

The FreeBSD Ports Collection.

Important: If you do not want to update the whole of ports-all (the whole ports tree), but use one of the subcollections listed below, make sure that you always update the ports-base subcollection! Whenever something changes in the ports build infrastructure represented by ports-base, it is virtually certain that those changes will be used by “real” ports real soon. Thus, if you only update the “real” ports and they use some of the new features, there is a very high chance that their build will fail with some mysterious error message. The very first thing to do in this case is to make sure that your ports-base subcollection is up to date.

Important: If you are going to be building your own local copy of ports/INDEX, you must accept ports-all (the whole ports tree). Building ports/INDEX with a partial tree is not supported. See the FAQ.

ports-accessibility release=cvs

Software to help disabled users.

ports-arabic release=cvs

Arabic language support.

ports-archivers release=cvs

Archiving tools.

ports-astro release=cvs

Astronomical ports.

ports-audio release=cvs

Sound support.

ports-base release=cvs

The Ports Collection build infrastructure - various files located in the Mk/ and Tools/ subdirectories of /usr/ports.

Note: Please see the important warning above: you should always update this subcollection, whenever you update any part of the FreeBSD Ports Collection!

ports-benchmarks release=cvs

Benchmarks.

ports-biology release=cvs

Biology.

ports-cad release=cvs

Computer aided design tools.

ports-chinese release=cvs

Chinese language support.

ports-comms release=cvs

Communication software.

ports-converters release=cvs

character code converters.

ports-databases release=cvs

Databases.

ports-deskutils release=cvs

Things that used to be on the desktop before computers were invented.

ports-devel release=cvs

Development utilities.

ports-dns release=cvs

DNS related software.

ports-editors release=cvs

Editors.

ports-emulators release=cvs

Emulators for other operating systems.

ports-finance release=cvs

Monetary, financial and related applications.

ports-ftp release=cvs

FTP client and server utilities.

ports-games release=cvs

Games.

ports-german release=cvs

German language support.

ports-graphics release=cvs

Graphics utilities.

ports-hebrew release=cvs

Hebrew language support.

ports-hungarian release=cvs

Hungarian language support.

ports-irc release=cvs

Internet Relay Chat utilities.

ports-japanese release=cvs

Japanese language support.

ports-java release=cvs

Java™ utilities.

ports-korean release=cvs

Korean language support.

ports-lang release=cvs

Programming languages.

ports-mail release=cvs

Mail software.

ports-math release=cvs

Numerical computation software.

ports-mbone release=cvs

MBone applications.

ports-misc release=cvs

Miscellaneous utilities.

ports-multimedia release=cvs

Multimedia software.

ports-net release=cvs

Networking software.

ports-net-im release=cvs

Instant messaging software.

ports-net-mgmt release=cvs

Network management software.

ports-net-p2p release=cvs

Peer to peer networking.

ports-news release=cvs

USENET news software.

ports-palm release=cvs

Software support for Palm™ series.

ports-polish release=cvs

Polish language support.

ports-ports-mgmt release=cvs

Utilities to manage ports and packages.

ports-portuguese release=cvs

Portuguese language support.

ports-print release=cvs

Printing software.

ports-russian release=cvs

Russian language support.

ports-science release=cvs

Science.

ports-security release=cvs

Security utilities.

ports-shells release=cvs

Command line shells.

ports-sysutils release=cvs

System utilities.

ports-textproc release=cvs

text processing utilities (does not include desktop publishing).

ports-ukrainian release=cvs

Ukrainian language support.

ports-vietnamese release=cvs

Vietnamese language support.

ports-www release=cvs

Software related to the World Wide Web.

ports-x11 release=cvs

Ports to support the X window system.

ports-x11-clocks release=cvs

X11 clocks.

ports-x11-drivers release=cvs

X11 drivers.

ports-x11-fm release=cvs

X11 file managers.

ports-x11-fonts release=cvs

X11 fonts and font utilities.

ports-x11-toolkits release=cvs

X11 toolkits.

ports-x11-servers release=cvs

X11 servers.

ports-x11-themes release=cvs

X11 themes.

ports-x11-wm release=cvs

X11 window managers.

projects-all release=cvs

Sources for the FreeBSD projects repository.

src-all release=cvs

The main FreeBSD sources, including the cryptography code.

src-base release=cvs

Miscellaneous files at the top of /usr/src.

src-bin release=cvs

User utilities that may be needed in single-user mode (/usr/src/bin).

src-cddl release=cvs

Utilities and libraries covered by the CDDL license (/usr/src/cddl).

src-contrib release=cvs

Utilities and libraries from outside the FreeBSD project, used relatively unmodified (/usr/src/contrib).

src-crypto release=cvs

Cryptography utilities and libraries from outside the FreeBSD project, used relatively unmodified (/usr/src/crypto).

src-eBones release=cvs

Kerberos and DES (/usr/src/eBones). Not used in current releases of FreeBSD.

src-etc release=cvs

System configuration files (/usr/src/etc).

src-games release=cvs

Games (/usr/src/games).

src-gnu release=cvs

Utilities covered by the GNU Public License (/usr/src/gnu).

src-include release=cvs

Header files (/usr/src/include).

src-kerberos5 release=cvs

Kerberos5 security package (/usr/src/kerberos5).

src-kerberosIV release=cvs

KerberosIV security package (/usr/src/kerberosIV).

src-lib release=cvs

Libraries (/usr/src/lib).

src-libexec release=cvs

System programs normally executed by other programs (/usr/src/libexec).

src-release release=cvs

Files required to produce a FreeBSD release (/usr/src/release).

src-rescue release=cvs

Statically linked programs for emergency recovery; see rescue(8) (/usr/src/rescue).

src-sbin release=cvs

System utilities for single-user mode (/usr/src/sbin).

src-secure release=cvs

Cryptographic libraries and commands (/usr/src/secure).

src-share release=cvs

Files that can be shared across multiple systems (/usr/src/share).

src-sys release=cvs

The kernel (/usr/src/sys).

src-sys-crypto release=cvs

Kernel cryptography code (/usr/src/sys/crypto).

src-tools release=cvs

Various tools for the maintenance of FreeBSD (/usr/src/tools).

src-usrbin release=cvs

User utilities (/usr/src/usr.bin).

src-usrsbin release=cvs

System utilities (/usr/src/usr.sbin).

www release=cvs

The sources for the FreeBSD WWW site.

distrib release=self

The CVSup server's own configuration files. Used by CVSup mirror sites.

gnats release=current

The GNATS bug-tracking database.

mail-archive release=current

FreeBSD mailing list archive.

www release=current

The pre-processed FreeBSD WWW site files (not the source files). Used by WWW mirror sites.

A.6.6 For More Information

For the CVSup FAQ and other information about CVSup, see The CVSup Home Page.

Most FreeBSD-related discussion of CVSup takes place on the FreeBSD technical discussions mailing list. New versions of the software are announced there, as well as on the FreeBSD announcements mailing list.

For questions or bug reports about CVSup take a look at the CVSup FAQ.

A.6.7 CVSup Sites

CVSup servers for FreeBSD are running at the following sites:

Central Servers, Primary Mirror Sites, Argentina, Armenia, Australia, Austria, Brazil, Bulgaria, Canada, China, Costa Rica, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Indonesia, Ireland, Israel, Italy, Japan, Korea, Kuwait, Kyrgyzstan, Latvia, Lithuania, Netherlands, New Zealand, Norway, Philippines, Poland, Portugal, Romania, Russia, San Marino, Singapore, Slovak Republic, Slovenia, South Africa, Spain, Sweden, Switzerland, Taiwan, Thailand, Turkey, Ukraine, United Kingdom, USA.

(as of 2009/11/04 17:21:16 UTC)

Central Servers
  • cvsup.FreeBSD.org

Primary Mirror Sites
  • cvsup1.FreeBSD.org

  • cvsup2.FreeBSD.org

  • cvsup3.FreeBSD.org

  • cvsup4.FreeBSD.org

  • cvsup5.FreeBSD.org

  • cvsup6.FreeBSD.org

  • cvsup7.FreeBSD.org

  • cvsup8.FreeBSD.org

  • cvsup9.FreeBSD.org

  • cvsup10.FreeBSD.org

  • cvsup11.FreeBSD.org

  • cvsup12.FreeBSD.org

  • cvsup13.FreeBSD.org

  • cvsup14.FreeBSD.org

  • cvsup15.FreeBSD.org

  • cvsup16.FreeBSD.org

  • cvsup18.FreeBSD.org

Argentina
  • cvsup.ar.FreeBSD.org

Armenia
  • cvsup1.am.FreeBSD.org

Australia
  • cvsup.au.FreeBSD.org

Austria
  • cvsup.at.FreeBSD.org

  • cvsup2.at.FreeBSD.org

Brazil
  • cvsup.br.FreeBSD.org

  • cvsup2.br.FreeBSD.org

  • cvsup3.br.FreeBSD.org

  • cvsup4.br.FreeBSD.org

  • cvsup5.br.FreeBSD.org

Bulgaria
  • cvsup.bg.FreeBSD.org

Canada
  • cvsup1.ca.FreeBSD.org

China
  • cvsup.cn.FreeBSD.org

  • cvsup2.cn.FreeBSD.org

  • cvsup3.cn.FreeBSD.org

  • cvsup4.cn.FreeBSD.org

  • cvsup5.cn.FreeBSD.org

Costa Rica
  • cvsup1.cr.FreeBSD.org

Czech Republic
  • cvsup.cz.FreeBSD.org

Denmark
  • cvsup.dk.FreeBSD.org

  • cvsup2.dk.FreeBSD.org

Estonia
  • cvsup.ee.FreeBSD.org

Finland
  • cvsup.fi.FreeBSD.org

  • cvsup2.fi.FreeBSD.org

France
  • cvsup.fr.FreeBSD.org

  • cvsup1.fr.FreeBSD.org

  • cvsup2.fr.FreeBSD.org

  • cvsup3.fr.FreeBSD.org

  • cvsup4.fr.FreeBSD.org

  • cvsup5.fr.FreeBSD.org

  • cvsup8.fr.FreeBSD.org

Germany
  • cvsup.de.FreeBSD.org

  • cvsup2.de.FreeBSD.org

  • cvsup3.de.FreeBSD.org

  • cvsup4.de.FreeBSD.org

  • cvsup5.de.FreeBSD.org

  • cvsup6.de.FreeBSD.org

  • cvsup7.de.FreeBSD.org

  • cvsup8.de.FreeBSD.org

Greece
  • cvsup.gr.FreeBSD.org

  • cvsup2.gr.FreeBSD.org

Hungary
  • cvsup.hu.FreeBSD.org

Iceland
  • cvsup.is.FreeBSD.org

Indonesia
  • cvsup.id.FreeBSD.org

Ireland
  • cvsup.ie.FreeBSD.org

  • cvsup2.ie.FreeBSD.org

Israel
  • cvsup.il.FreeBSD.org

Italy
  • cvsup.it.FreeBSD.org

Japan
  • cvsup.jp.FreeBSD.org

  • cvsup2.jp.FreeBSD.org

  • cvsup3.jp.FreeBSD.org

  • cvsup4.jp.FreeBSD.org

  • cvsup5.jp.FreeBSD.org

  • cvsup6.jp.FreeBSD.org

Korea
  • cvsup.kr.FreeBSD.org

  • cvsup2.kr.FreeBSD.org

  • cvsup3.kr.FreeBSD.org

Kuwait
  • cvsup1.kw.FreeBSD.org

Kyrgyzstan
  • cvsup.kg.FreeBSD.org

Latvia
  • cvsup.lv.FreeBSD.org

  • cvsup2.lv.FreeBSD.org

Lithuania
  • cvsup.lt.FreeBSD.org

  • cvsup2.lt.FreeBSD.org

  • cvsup3.lt.FreeBSD.org

Netherlands
  • cvsup.nl.FreeBSD.org

  • cvsup2.nl.FreeBSD.org

  • cvsup3.nl.FreeBSD.org

New Zealand
  • cvsup.nz.FreeBSD.org

  • cvsup2.nz.FreeBSD.org

Norway
  • cvsup.no.FreeBSD.org

Philippines
  • cvsup1.ph.FreeBSD.org

Poland
  • cvsup.pl.FreeBSD.org

  • cvsup2.pl.FreeBSD.org

  • cvsup3.pl.FreeBSD.org

Portugal
  • cvsup.pt.FreeBSD.org

  • cvsup2.pt.FreeBSD.org

  • cvsup3.pt.FreeBSD.org

Romania
  • cvsup.ro.FreeBSD.org

  • cvsup1.ro.FreeBSD.org

  • cvsup2.ro.FreeBSD.org

  • cvsup3.ro.FreeBSD.org

Russia
  • cvsup.ru.FreeBSD.org

  • cvsup2.ru.FreeBSD.org

  • cvsup3.ru.FreeBSD.org

  • cvsup4.ru.FreeBSD.org

  • cvsup5.ru.FreeBSD.org

  • cvsup6.ru.FreeBSD.org

  • cvsup7.ru.FreeBSD.org

San Marino
  • cvsup.sm.FreeBSD.org

Singapore
  • cvsup.sg.FreeBSD.org

Slovak Republic
  • cvsup.sk.FreeBSD.org

Slovenia
  • cvsup.si.FreeBSD.org

  • cvsup2.si.FreeBSD.org

South Africa
  • cvsup.za.FreeBSD.org

  • cvsup2.za.FreeBSD.org

Spain
  • cvsup.es.FreeBSD.org

  • cvsup2.es.FreeBSD.org

  • cvsup3.es.FreeBSD.org

Sweden
  • cvsup.se.FreeBSD.org

  • cvsup3.se.FreeBSD.org

Switzerland
  • cvsup.ch.FreeBSD.org

Taiwan
  • cvsup.tw.FreeBSD.org

  • cvsup3.tw.FreeBSD.org

  • cvsup4.tw.FreeBSD.org

  • cvsup5.tw.FreeBSD.org

  • cvsup6.tw.FreeBSD.org

  • cvsup7.tw.FreeBSD.org

  • cvsup8.tw.FreeBSD.org

  • cvsup9.tw.FreeBSD.org

  • cvsup10.tw.FreeBSD.org

  • cvsup11.tw.FreeBSD.org

  • cvsup12.tw.FreeBSD.org

  • cvsup13.tw.FreeBSD.org

  • cvsup14.tw.FreeBSD.org

Thailand
  • cvsup.th.FreeBSD.org

Turkey
  • cvsup.tr.FreeBSD.org

  • cvsup2.tr.FreeBSD.org

Ukraine
  • cvsup2.ua.FreeBSD.org

  • cvsup3.ua.FreeBSD.org

  • cvsup4.ua.FreeBSD.org

  • cvsup5.ua.FreeBSD.org

  • cvsup6.ua.FreeBSD.org

  • cvsup7.ua.FreeBSD.org

United Kingdom
  • cvsup.uk.FreeBSD.org

  • cvsup2.uk.FreeBSD.org

  • cvsup3.uk.FreeBSD.org

  • cvsup4.uk.FreeBSD.org

USA
  • cvsup1.us.FreeBSD.org

  • cvsup2.us.FreeBSD.org

  • cvsup3.us.FreeBSD.org

  • cvsup4.us.FreeBSD.org

  • cvsup5.us.FreeBSD.org

  • cvsup6.us.FreeBSD.org

  • cvsup7.us.FreeBSD.org

  • cvsup8.us.FreeBSD.org

  • cvsup9.us.FreeBSD.org

  • cvsup10.us.FreeBSD.org

  • cvsup11.us.FreeBSD.org

  • cvsup12.us.FreeBSD.org

  • cvsup13.us.FreeBSD.org

  • cvsup14.us.FreeBSD.org

  • cvsup15.us.FreeBSD.org

  • cvsup16.us.FreeBSD.org

  • cvsup18.us.FreeBSD.org

This, and other documents, can be downloaded from ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/.

For questions about FreeBSD, read the documentation before contacting <questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.