Table of Contents
CFLAGS
?This section contains hints, tips & tricks on special things in pkgsrc that we didn't find a better place for in the previous chapters, and it contains items for both pkgsrc users and developers.
The following mailing lists may be of interest to pkgsrc users:
pkgsrc-users: This is a general purpose list for most issues regarding pkgsrc, regardless of platform, e.g. soliciting user help for pkgsrc configuration, unexpected build failures, using particular packages, upgrading pkgsrc installations, questions regarding the pkgsrc release branches, etc. General announcements or proposals for changes that impact the pkgsrc user community, e.g. major infrastructure changes, new features, package removals, etc., may also be posted.
pkgsrc-bulk: A list where the results of pkgsrc bulk builds are sent and discussed.
pkgsrc-changes: This list is for those who are interested in getting a commit message for every change committed to pkgsrc. It is also available in digest form, meaning one daily message containing all commit messages for changes to the package source tree in that 24 hour period.
To subscribe, do:
%
echo subscribelistname
| mail majordomo@NetBSD.org
Archives for all these mailing lists are available from https://mail-index.NetBSD.org/.
The directory pkgsrc/pkgtools
contains
a number of useful utilities for both users and developers of pkgsrc. This
section attempts only to make the reader aware of some of the utilities and when
they might be useful, and not to duplicate the documentation that comes
with each package.
Utilities used by pkgsrc (automatically installed when needed):
pkgtools/x11-links
:
Symlinks for use by buildlink.
OS tool augmentation (automatically installed when needed):
pkgtools/digest
:
Calculates various kinds of checksums (including SHA3).
pkgtools/libnbcompat
:
Compatibility library for pkgsrc tools.
pkgtools/mtree
: Installed on
non-BSD systems due to lack of native mtree.
pkgtools/pkg_install
:
Up-to-date replacement for
/usr/sbin/pkg_install
, or for use on operating
systems where pkg_install is not present.
Utilities used by pkgsrc (not automatically installed):
pkgtools/pkg_tarup
:
Create a binary package from an
already-installed package. Used by make replace to
save the old package.
pkgtools/dfdisk
:
Adds extra functionality to pkgsrc, allowing it to fetch distfiles
from multiple locations. It currently supports the following
methods: multiple CD-ROMs and network FTP/HTTP connections.
devel/cpuflags
: Determine
the best compiler flags to optimise code for your current
CPU and compiler.
Utilities for keeping track of installed packages, being up to date, etc:
pkgtools/pkgin
: A package
update tool similar to apt(1). Download, install, and upgrade
binary packages easily.
pkgtools/pkg_chk
: Reports on
packages whose installed versions do not match the latest pkgsrc
entries.
pkgtools/pkgdep
: Makes
dependency graphs of packages, to aid in choosing a strategy for
updating.
pkgtools/pkgdepgraph
: Makes
graphs from the output of pkgtools/pkgdep
(uses graphviz).
pkgtools/pkglint
: The
pkglint(1) program checks a pkgsrc entry for errors.
pkgtools/lintpkgsrc
: The lintpkgsrc(1) program
does various checks on the complete pkgsrc system.
pkgtools/pkgsurvey
: Report what
packages you have installed.
Utilities for people maintaining or creating individual packages:
pkgtools/pkgdiff
: Automate
making and maintaining patches for a package (includes pkgdiff,
pkgvi, mkpatches, etc.).
pkgtools/url2pkg
: Aids in
converting to pkgsrc.
Utilities for people maintaining pkgsrc (or: more obscure pkg utilities)
pkgtools/pkg_comp
: Build
packages in a chrooted area.
pkgtools/libkver
: Spoof
kernel version for chrooted cross builds.
To install packages from source as a non-root user, download pkgsrc as described in Chapter 3, Where to get pkgsrc and how to keep it up-to-date, cd into that directory and run the command ./bootstrap/bootstrap --unprivileged.
This will install the binary part of pkgsrc to
~/pkg
and put the pkgsrc configuration mk.conf
into ~/pkg/etc
.
For more details, see mk/unprivileged.mk
.
By default, resuming transfers in pkgsrc is disabled, but you can
enable this feature by adding the option
PKG_RESUME_TRANSFERS=YES
into
mk.conf
. If, during a fetch step, an incomplete
distfile is found, pkgsrc will try to resume it.
You can also
use a different program than the platform default program by changing the
FETCH_USING
variable. You can specify the program by
using of ftp, fetch, wget or curl. Alternatively, fetching can be disabled
by using the value manual. A value of custom disables the system defaults
and dependency tracking for the fetch program. In that case you have to
provide FETCH_CMD
, FETCH_BEFORE_ARGS
,
FETCH_RESUME_ARGS
, FETCH_OUTPUT_ARGS
,
FETCH_AFTER_ARGS
.
For example, if you want to use
wget
to download, you'll have to use something
like:
FETCH_USING= wget
If you want to use modular X.org from pkgsrc instead of your system's own X11
(/usr/X11R6
, /usr/openwin
, ...)
you will have to add the following line into
mk.conf
:
X11_TYPE=modular
If you are sitting behind a firewall which does not allow direct connections to Internet hosts (i.e. non-NAT), you may specify the relevant proxy hosts. This is done using an environment variable in the form of a URL, e.g. in Amdahl, the machine “orpheus.amdahl.com” is one of the firewalls, and it uses port 80 as the proxy port number. So the proxy environment variables are:
ftp_proxy=ftp://orpheus.amdahl.com:80/ http_proxy=http://orpheus.amdahl.com:80/
Some fetch tools are not prepared to support HTTPS by default (for example, the one in NetBSD 6.0), or the one installed by the pkgsrc bootstrap (to avoid an openssl dependency that low in the dependency graph).
Usually you won't notice, because distribution files are
mirrored weekly to “ftp.NetBSD.org”, but that might not
be often enough if you are following pkgsrc-current. In that case, set
FETCH_USING
in your mk.conf
file to
“curl” or “wget”, which are both compiled
with HTTPS support by default. Of course, these tools need to be
installed before you can use them this way.
This depends on which utility is used to retrieve distfiles. From
bsd.pkg.mk
, FETCH_CMD
is assigned
the first available command from the following list:
${LOCALBASE}/bin/ftp
/usr/bin/ftp
On a default NetBSD installation, this will be
/usr/bin/ftp
, which automatically tries passive
connections first, and falls back to active connections if the server
refuses to do passive. For the other tools, add the following to your
mk.conf
file:
PASSIVE_FETCH=1
.
Having that option present will prevent
/usr/bin/ftp
from falling back to active
transfers.
You would like to download all the distfiles in a single batch from work or university, where you can't run a make fetch. There is an archive of distfiles on ftp.NetBSD.org, but downloading the entire directory may not be appropriate.
The answer here is to do a make fetch-list in
/usr/pkgsrc
or one of its subdirectories, carry the
resulting list to your machine at work/school and use it there. If you
don't have a NetBSD-compatible ftp(1) (like tnftp) at work, don't
forget to set FETCH_CMD
to something that fetches a
URL:
At home:
%
cd /usr/pkgsrc
%
make fetch-list FETCH_CMD=wget DISTDIR=/tmp/distfiles >/tmp/fetch.sh
%
scp /tmp/fetch.sh work:/tmp
At work:
%
sh /tmp/fetch.sh
then tar up /tmp/distfiles
and take it
home.
If you have a machine running NetBSD, and you want to get all distfiles (even ones that aren't for your machine architecture), you can do so by using the above-mentioned make fetch-list approach, or fetch the distfiles directly by running:
%
make mirror-distfiles
If you even decide to ignore
NO_{SRC,BIN}_ON_{FTP,CDROM}
, then you can get everything
by running:
%
make fetch NO_SKIP=yes
When compiling the pkgtools/pkg_install
package, you get the error from make that it doesn't know how to make
/usr/share/tmac/tmac.andoc
? This indicates that
you don't have installed the “text” set (nroff, ...) from
the NetBSD base distribution on your machine. It is recommended to do
that to format man pages.
In the case of the pkgtools/pkg_install
package, you
can get away with setting NOMAN=YES
either in the
environment or in mk.conf
.
You didn't install the compiler set, comp.tgz
,
when you installed your NetBSD machine. Please get and install it, by
extracting it in /
:
#
cd /
#
tar --unlink -zxvpf .../comp.tgz
comp.tgz
is part of every NetBSD release. Get
the one that corresponds to your release (determine via uname
-r).
When installing packages as non-root user and using the just-in-time
su(1) feature of pkgsrc, it can become annoying to type in the root
password for each required package installed. To avoid this, the sudo
package can be used, which does password caching over a limited time. To
use it, install sudo or
priv (either as binary package, from
security/priv
, or
security/sudo
) and then put the
following into your mk.conf
, somewhere
after the definition of the
LOCALBASE
variable:
.if exists(${LOCALBASE}/bin/sudo) SU_CMD= ${LOCALBASE}/bin/sudo /bin/sh -c .endif
As the system administrator, you can choose where configuration files
are installed. The default settings make all these files go into
${PREFIX}/etc
or some of its subdirectories; this may
be suboptimal depending on your expectations (e.g., a read-only,
NFS-exported PREFIX
with a need of per-machine
configuration of the provided packages).
In order to change the defaults, you can modify the
PKG_SYSCONFBASE
variable (in
mk.conf
) to point to your preferred configuration
directory; some common examples include /etc
or
/etc/pkg
.
Furthermore, you can change this value on a per-package basis by
setting the PKG_SYSCONFDIR.${PKG_SYSCONFVAR}
variable.
PKG_SYSCONFVAR
's value usually matches the name of the
package you would like to modify, that is, the contents of
PKGBASE
.
Note that after changing these settings, you must rebuild and reinstall any affected packages.
Please be aware that there can often be bugs in third-party software,
and some of these bugs can leave a machine vulnerable to exploitation by
attackers. In an effort to lessen the exposure, the NetBSD packages team
maintains a database of known-exploits to packages which have at one time
been included in pkgsrc. The database can be downloaded automatically, and
a security audit of all packages installed on a system can take place. To
do this, refer to the following two tools (installed as part of the
pkgtools/pkg_install
package):
pkg_admin fetch-pkg-vulnerabilities, an easy way to download a list of the security vulnerabilities information. This list is kept up to date by the pkgsrc security team, and is distributed from the NetBSD ftp server:
https://ftp.NetBSD.org/pub/NetBSD/packages/vulns/pkg-vulnerabilities
pkg_admin audit, an easy way to audit the current machine, checking each known vulnerability. If a vulnerable package is installed, it will be shown by output to stdout, including a description of the type of vulnerability, and a URL containing more information.
Use of these tools is strongly recommended! See Section 5.1.6, “Checking for security vulnerabilities in installed packages” for instructions on how to automate checking and reporting.
If this database is installed, pkgsrc builds will use it to perform a security check before building any package.
When you add your own preferences to the
CFLAGS
variable in your
mk.conf
, these flags are passed in
environment variables to the ./configure
scripts and to make(1). Some package authors ignore the
CFLAGS
from the environment variable by
overriding them in the Makefile
s of their
package.
Currently there is no solution to this problem. If you
really need the package to use your CFLAGS
you should run make patch in the package
directory and then inspect any Makefile
and
Makefile.in
for whether they define
CFLAGS
explicitly. Usually you can remove
these lines. But be aware that some “smart”
programmers write so bad code that it only works for the
specific combination of CFLAGS
they have
chosen.
To find out where the CFLAGS are ignored, add the following lines to mk.conf
:
CPPFLAGS+= -Dpkgsrc___CPPFLAGS CFLAGS+= -Dpkgsrc___CFLAGS CXXFLAGS+= -Dpkgsrc___CXXFLAGS
Then run bmake show-all-configure show-all-build to see whether the above flags are passed to the actual build commands in general.
To find out whether the flags are passed to individual compiler
commands, have a look at the file work/.work.log
. In
most cases, the flags from the original command lines (the lines starting
with [*]
) are passed unmodified to the actual compiler
(the lines starting with <.>
). If the flag is
missing from the actual compiler command, it must have been removed by
the pkgsrc compiler wrappers.
Make sure that your copy of pkgsrc is consistent. A case that occurs often is that people only update pkgsrc in parts, because of performance reasons. Since pkgsrc is one large system, not a collection of many small systems, there are sometimes changes that only work when the whole pkgsrc tree is updated.
Make sure that you don't have any CVS conflicts. Search for “<<<<<<” or “>>>>>>” in all your pkgsrc files.
Make sure that you don't have old copies of the packages extracted. Run make clean clean-depends to verify this.
If you are a package developer who wants to invest some work, have a look at Chapter 21, Making your package work.
If the problem still exists, write a mail to the
pkgsrc-users
mailing list.
You have modified a file from pkgsrc, and someone else has
modified that same file afterwards in the CVS repository. Both changes
are in the same region of the file, so when you updated pkgsrc, the
cvs
command marked the conflicting changes in the
file. Because of these markers, the file is no longer a valid
Makefile
.
Have a look at that file, and if you don't need your local changes anymore, you can remove that file and run cvs -q update -dP in that directory to download the current version.