Table of Contents
When you find a package that is not yet in pkgsrc, you most likely have a URL from where you can download the source code. Starting with this URL, creating a package involves only a few steps.
In your mk.conf, set PKG_DEVELOPER=yes to
enable the basic quality checks.
Install the package meta-pkgs/pkg_developer, which among others will
install the utilities url2pkg,
pkglint, pkgvi and
mkpatches:
$cd /usr/pkgsrc$(cd meta-pkgs/pkg_developer && bmake update)
Choose one of the top-level directories as the category in
which you want to place your package. You can also create a directory of
your own (maybe called local). Change into that
category directory:
$cdcategory
Run the program url2pkg, passing as
argument the URL of the distribution file (in most cases a
.tar.gz file). This will download the distribution
file and create the necessary files of the package, based on what's in
the distribution file:
$url2pkghttps://www.example.org/packages/package-1.0.tar.gz
Examine the extracted files to determine the dependencies of
your package. Ideally, this is mentioned in some
README file, but things may differ. For each of
these dependencies, look where it exists in pkgsrc, and if there is a
file called buildlink3.mk in that directory, add a
line to your package Makefile which includes that
file just before the last line. If the
buildlink3.mk file does not exist, it must be
created first. The buildlink3.mk file makes sure
that the package's include files and libraries are provided.
If you just need binaries from a dependent package, add a
DEPENDS line to the Makefile, which specifies the
version of the dependency and where it can be found in pkgsrc. This line
should be placed in the third paragraph. If the dependency is only
needed for building the package, but not when using it, use
TOOL_DEPENDS or BUILD_DEPENDS
instead of DEPENDS.
The difference between TOOL_DEPENDS and
BUILD_DEPENDS occurs when cross-compiling:
TOOL_DEPENDS are native
packages, i.e. packages for the platform where the package is built;
BUILD_DEPENDS are target
packages, i.e. packages for the platform for which the package
is built. There is also TEST_DEPENDS, which
specifies a dependency used only for testing the resulting package
built, using the upstream project's included test suite, on the native
platform.
Your package may then look like this:
[...] TOOL_DEPENDS+= libxslt-[0-9]*:../../textproc/libxslt DEPENDS+= screen-[0-9]*:../../misc/screen DEPENDS+= screen>=4.0:../../misc/screen [...] .include "../../category/package/buildlink3.mk" .include "../../devel/glib2/buildlink3.mk" .include "../../mk/bsd.pkg.mk"
Run pkglint to see what things still need to be done to make your package a “good” one. If you don't know what pkglint's warnings want to tell you, try pkglint --explain or pkglint -e, which outputs additional explanations.
In many cases the package is not yet ready to build. You can find instructions for the most common cases in the next section, Section 14.1, “Common types of packages”. After you have followed the instructions over there, you can hopefully continue here.
Run bmake clean to clean the working
directory from the extracted files. Besides these files, a lot of cache
files and other system information have been saved in the working
directory, which may have become outdated after you edited the
Makefile.
Now, run bmake to build the package. For the various things that can go wrong in this phase, consult Chapter 21, Making your package work.
If the extracted files from the package need to be fixed, run multiple rounds of these commands:
$bmake$pkgvi ${WRKSRC}/some/file/that/does/not/compile$mkpatches$bmake mps$bmake clean
When the package builds fine, the next step is to install the package. Run bmake install and hope that everything works.
Up to now, the file PLIST, which
contains a list of the files that are installed by the package, is
nearly empty. Run bmake print-PLIST
>PLIST to generate a probably correct list. Check
the file using your preferred text editor to see if the list of
files looks plausible.
Run pkglint again to see if the generated
PLIST contains garbage or not.
When you ran bmake install, the package
had been registered in the database of installed files, but with an
empty list of files. To fix this, run bmake deinstall
and bmake install again. Now the package is
registered with the list of files from
PLIST.
Run bmake clean update to run everything from above again in a single step, making sure that the PLIST is correct and the whole package is created as intended.
Run pkglint to see if there's anything left to do.
Commit the package to pkgsrc-wip or main pkgsrc; see Chapter 23, Submitting and Committing.
Python modules and programs packages are easily created using a set of predefined variables.
If some Python versions are not supported by the software, set the
PYTHON_VERSIONS_INCOMPATIBLE variable to the Python versions
that are not supported, e.g.
PYTHON_VERSIONS_INCOMPATIBLE= 27
If the packaged software is a Python module, include
../../lang/python/wheel.mk. Note per PEP-518,
the minimum and default requirements to create .whl files are
devel/py-setuptools plus devel/py-wheel; however, there are various other
possible tools that projects can use. Thus inclusion of
wheel.mk does not imply these defaults are
defined as TOOL_DEPENDS. Whatever the project
specifically requires as packaging tools must be defined in the
package Makefile.
The package directory should be called
“py-software” and PKGNAME should be set to
“${PYPKGPREFIX}-${DISTNAME}”, e.g.
DISTNAME= foopymodule-1.2.10
PKGNAME= ${PYPKGPREFIX}-${DISTNAME}
For software in PyPi, the name should match what PyPi specifies for "pip install software".
If it is an application, include
“../../lang/python/application.mk”.
In order to correctly set the path to the Python interpreter, use the
REPLACE_PYTHON variable and set it to the list of files
(paths relative to WRKSRC) that must be corrected.
For example:
REPLACE_PYTHON= *.py
Simple R packages from CRAN
are handled automatically by R2pkg, which is
available in pkgtools/R2pkg.
Individual packages (and optionally their dependencies) may be created
and updated. R packages generally follow the same form, and most of
the relevant information needed is contained in a
DESCRIPTION file as part of each R package on
CRAN.
Consequently, R2pkg downloads that information and
creates or updates a package in the canonical form. The resulting
package should be reviewed for correctness.
TeXlive packages from CTAN are handled automatically by
texlive2pkg, which is available in pkgtools/texlive2pkg.
If the TeXlive package name is not known, it may be useful to search CTAN. A “Contained in” field on the package page typically identifies the basename of the package file in the TeXlive archive.
If the TeXlive package name is known, download the files from
the TeXlive
archive. For package foo, you will need
to download foo.tar.xz. Most TeXlive packages
also have associated documentation packages, so download
foo.doc.tar.xz at the same time. These files
should be placed in the appropriate category directory, which is often
but not always print. Then run the following
command in the category directory.
texlive2pkg foo.tar.xz foo.doc.tar.xz
This will create two packages, tex-foo and
tex-foo-doc. Be sure to check that both packages
are correct.
Finally, CTAN currently does not include version information in package filenames and changes their contents periodically when updates occur. Consequently, pkgsrc avoids downloading distfiles directly from CTAN and instead relies on the pkgsrc archives. For each new or updated TeXlive package, e.g., the main one and the corresponding documentation, upload the distfiles with the following command in each package directory.
make upload-distfiles
Looking at the file pkgsrc/doc/TODO, I saw
that the “nvu” package has not yet been imported into
pkgsrc. As the description says it has to do with the web, the obvious
choice for the category is “www”.
$mkdir www/nvu$cd www/nvu
The web site says that the sources are available as a tar file, so I fed that URL to the url2pkg program:
$ url2pkg http://cvs.nvu.com/download/nvu-1.0-sources.tar.bz2
My editor popped up, and I added a PKGNAME line
below the DISTNAME line, as the package name should
not have the word “sources” in it. I also filled in the
MAINTAINER, HOMEPAGE and
COMMENT fields. Then the package
Makefile looked like that:
# $NetBSD $ # DISTNAME= nvu-1.0-sources PKGNAME= nvu-1.0 CATEGORIES= www MASTER_SITES= http://cvs.nvu.com/download/ EXTRACT_SUFX= .tar.bz2 MAINTAINER= rillig@NetBSD.org HOMEPAGE= http://cvs.nvu.com/ COMMENT= Web Authoring System # url2pkg-marker (please do not remove this line.) .include "../../mk/bsd.pkg.mk"
On the first line of output above, an artificial space has been added between NetBSD and $, this is a workaround to prevent CVS expanding to the filename of the guide.
Then, I quit the editor and watched pkgsrc downloading a large source archive:
url2pkg> Running "make makesum" ... => Required installed package digest>=20010302: digest-20060826 found => Fetching nvu-1.0-sources.tar.bz2 Requesting http://cvs.nvu.com/download/nvu-1.0-sources.tar.bz2 100% |*************************************| 28992 KB 150.77 KB/s00:00 ETA 29687976 bytes retrieved in 03:12 (150.77 KB/s) url2pkg> Running "make extract" ... => Required installed package digest>=20010302: digest-20060826 found => Checksum SHA1 OK for nvu-1.0-sources.tar.bz2 => Checksum RMD160 OK for nvu-1.0-sources.tar.bz2 work.bacc -> /tmp/roland/pkgsrc/www/nvu/work.bacc ===> Installing dependencies for nvu-1.0 ===> Overriding tools for nvu-1.0 ===> Extracting for nvu-1.0 url2pkg> Adjusting the Makefile. Remember to correct CATEGORIES, HOMEPAGE, COMMENT, and DESCR when you're done! Good luck! (See pkgsrc/doc/pkgsrc.txt for some more help :-)
Now that the package has been extracted, let's see what's inside
it. The package has a README.txt, but that only
says something about mozilla, so it's probably useless for seeing what
dependencies this package has. But since there is a GNU configure script
in the package, let's hope that it will complain about everything it
needs.
$ bmake
=> Required installed package digest>=20010302: digest-20060826 found
=> Checksum SHA1 OK for nvu-1.0-sources.tar.bz2
=> Checksum RMD160 OK for nvu-1.0-sources.tar.bz2
===> Patching for nvu-1.0
===> Creating toolchain wrappers for nvu-1.0
===> Configuring for nvu-1.0
[...]
configure: error: Perl 5.004 or higher is required.
[...]
WARNING: Please add USE_TOOLS+=perl to the package Makefile.
[...]
That worked quite well. So I opened the package Makefile in my
editor, and since it already has a USE_TOOLS line, I
just appended “perl” to it. Since the dependencies of the
package have changed now, and since a perl wrapper is automatically
installed in the “tools” phase, I need to build the package
from scratch.
$bmake clean ===> Cleaning for nvu-1.0$bmake [...] *** /tmp/roland/pkgsrc/www/nvu/work.bacc/.tools/bin/make is not \ GNU Make. You will not be able to build Mozilla without GNU Make. [...]
So I added “gmake” to the
USE_TOOLS line and tried again (from scratch).
[...] checking for GTK - version >= 1.2.0... no *** Could not run GTK test program, checking why... [...]
Now to the other dependencies. The first question is: Where is the GTK package hidden in pkgsrc?
$echo ../../*/gtk* [many packages ...]$echo ../../*/gtk ../../x11/gtk$echo ../../*/gtk2 ../../x11/gtk2$echo ../../*/gtk2/bui* ../../x11/gtk2/buildlink3.mk
The first try was definitely too broad. The second one had exactly one result, which is very good. But there is one pitfall with GNOME packages. Before GNOME 2 had been released, there were already many GNOME 1 packages in pkgsrc. To be able to continue to use these packages, the GNOME 2 packages were imported as separate packages, and their names usually have a “2” appended. So I checked whether this was the case here, and indeed it was.
Since the GTK2 package has a buildlink3.mk
file, adding the dependency is very easy. I just inserted an
.include line before the last line of the package
Makefile, so that it now looks like this:
[...] .include "../../x11/gtk2/buildlink3.mk" .include "../../mk/bsd.pkg.mk
After another bmake clean && bmake, the answer was:
[...] checking for gtk-config... /home/roland/pkg/bin/gtk-config checking for GTK - version >= 1.2.0... no *** Could not run GTK test program, checking why... *** The test program failed to compile or link. See the file config.log for the *** exact error that occured. This usually means GTK was incorrectly installed *** or that you have moved GTK since it was installed. In the latter case, you *** may want to edit the gtk-config script: /home/roland/pkg/bin/gtk-config configure: error: Test for GTK failed. [...]
In this particular case, the assumption that “every package
prefers GNOME 2” had been wrong. The first of the lines above
told me that this package really wanted to have the GNOME 1 version of
GTK. If the package had looked for GTK2, it would have looked for
pkg-config instead of gtk-config.
So I changed the x11/gtk2 to
x11/gtk in the package Makefile,
and tried again.
[...] cc -o xpidl.o -c -DOSTYPE=\"NetBSD3\" -DOSARCH=\"NetBSD\" [...] In file included from xpidl.c:42: xpidl.h:53:24: libIDL/IDL.h: No such file or directory In file included from xpidl.c:42: xpidl.h:132: error: parse error before "IDL_ns" [...]
The package still does not find all of its dependencies. Now the
question is: Which package provides the
libIDL/IDL.h header file?
$echo ../../*/*idl* ../../devel/py-idle ../../wip/idled ../../x11/acidlaunch$echo ../../*/*IDL* ../../net/libIDL
Let's take the one from the second try. So I included the
../../net/libIDL/buildlink3.mk file and tried
again. But the error didn't change. After digging through some of the
code, I concluded that the build process of the package was broken and
couldn't have ever worked, but since the Mozilla source tree is quite
large, I didn't want to fix it. So I added the following to the package
Makefile and tried again:
CPPFLAGS+= -I${BUILDLINK_PREFIX.libIDL}/include/libIDL-2.0
BUILDLINK_TRANSFORM+= l:IDL:IDL-2
The latter line is needed because the package expects the library
libIDL.so, but only
libIDL-2.so is available. So I told the compiler
wrapper to rewrite that on the fly.
The next problem was related to a recent change of the FreeType
interface. I looked up in www/seamonkey
which patch files were relevant for this issue and copied them to the
patches directory. Then I retried, fixed the
patches so that they applied cleanly and retried again. This time,
everything worked.