The NetBSD Foundation 2004 Annual Report
The NetBSD Foundation held its Annual Group Meeting of all members of the NetBSD Foundation, i.e. all NetBSD developers. From the more than 200 active developers, 65 made it to the meeting, which took place from 22:00 UTC to midnight on January 15th 2005. This is a report from the presentations of the various groups within the NetBSD project about their past achievements and future Goals. This report is mostly a verbatim version of the text presented in the online meeting, with a few places edited slightly, marked by "[...]".
1. Welcome and Intro remarks (Christos Zoulas)
2. Presentations from the Executive Committees
a. Technical exec (Alistair Crooks)
- core (Allen Briggs)
- security-officer (Daniel Carosone)
- releng (James Chacon)
- pkgsrc (Johnny Lam)
b. Communications exec (Tracy Di Marco White)
- www (Jan Schaumann)
c. Finance exec (Lex Wennmacher)
d. Membership exec (Thomas Klausner)
e. Administration exec (Tracy Di Marco White)
3. Any Other Business
4. Closing remarks (Christos Zoulas)
Christos Zoulas, president of The NetBSD Foundation, first introduced the members of the current Board of Directors:
-
Chris Demetriou and Tracy Di Marco White are members of the board,
-
Lex Wennmacher is a member of the board and Treasurer,
-
Luke Mewburn is a member of the board, Vice President and Secretary,
-
Christos Zoulas is member of the board and President of the NetBSD Foundation.
Christos thanked both personally and on behalf of the Board of Directors the two departing members Chris Demetriou and Luke Mewburn for their hard work during the past two years. Chris led the Foundation as President two years ago (followed by Christos in the past year) and Luke is the current secretary and during the past year was instrumental in organizing and overseeing the selection of our new Logo.
Next, Christos welcomed the two new Board members Alistair Crooks and Perry Metzger who will be replacing Chris and Luke and their term will begin at the end of this meeting.
Finally he thanked the chair of the Nomination Committee Jim Wise and its members, as well as Thomas Klausner who was our Election Administrator and Simon Gerraty who was the Election Validator for the Board Election.
Last, Christos thanked everyone involved in the Executive Committees who spend a lot of time, both visibly and behind the scenes to keep the Foundation running.
His congratulations went to the pkgsrc members for constantly improving a very high quality product, to the www team for keeping our website up-to-date, to the systems administration group for persevering despite all the hardware failures, and to the release engineering group for managing to push 2.0 out of the door. His personal thanks also went to Lex and Tracy for taking over a lot of what used to be his responsibilities and doing a better job than him.
Christos wished the new Board members Alistair and Perry a successful and fruitful Board term.
After he was done with the introductions Christos gave a bit of a personal perspective to his involvement with NetBSD that has been going on for the past 12 years:
``I joined NetBSD after lurking in the in the mailing
lists because I was truly impressed by the level of technical
expertise and the quality of the source code. I started as a
developer who did not know left from right on how anything in the
kernel worked (and some might say that I still don't) and ended up
being the President of the Foundation.
I am saying all this because I feel that our future depends on our
ability to attract, recruit, teach, organize, and guide new talented
people who are going to be the future of the project. As a volunteer
effort we depend on the generous time and resource offers of the
members our community.
We all have other obligations in our lives - both personal and
professional - which take precedence to NetBSD, so I understand how
difficult it is to allocate time on a consistent basis for a volunteer
program. I can say that my involvement has been very rewarding for me
both in improving my technical knowledge, and making me feel
appreciated for contributing to something that I do for fun and not
for financial gains.
I want to encourage everyone to think of ways to become more involved
with the running of the project. I value your opinion and need your
help to improve NetBSD. It is easy to sit back and complain about our
faults and deficiencies, but a lot harder to fix them. As a community
we excel in criticizing everything but when it comes to take
responsibility to making changes and fixing the problems only a few of
us step up to the plate.
Working on an Operating System is very challenging - more so now than
before. Ten years ago neither Windows or Linux were serious
competitors both in the functionality and stability axes. Now both
offer more features than we do, and they have behind them the
resources of very large commercial organizations. If that was not
enough competition, there is a plethora of other operating system
choices each trying to fill a niche. Even our close siblings FreeBSD
and OpenBSD have developed certain features we still lack.
Are we becoming irrelevant? Is it time to give up? I don't believe
that either is true. I see our competition facing challenges of their
own. Linux keeps re-writing major portions of the kernel and has
stability issues. It now depends on 3rd party vendors to integrate and
make stable releases of the code. FreeBSD took over the huge task to
implement fine grain SMP and after two years of effort they still
don't have a production quality system. OpenBSD is still touting its
security features but lacks the manpower to integrate major kernel
features such as UBC and address performance problems. Instead it
focuses in supporting and re-implementing major userland
utilities. The Windows release cycles keep getting longer and longer
and promised features keep getting postponed because of the increasing
complexity of the operating system. Sun is trying to keep Solaris
relevant by open-sourcing it, but nobody is certain of what is going
to be open-sourced and when. Apple's Darwin effort does not seem to be
producing any useful results, possibly because it is not complete, and
the open-source version of the tree is always behind the commercial
version.
So where does that leave us? What is the purpose of NetBSD?
I see a lot of opportunities ahead for us. We've managed to release
2.0 this year and we have integrated major features without serious
regressions or performance issues. We can leverage a lot of the work
in FreeBSD and OpenBSD to improve some of the areas we are lacking. We
have the cleanest source tree and our wealth of hardware support makes
us a very attractive porting target. Our cross-building capabilities
have made it possible to support the plethora of ports with much less
effort than before. Our BSD style license allows vendors to use our
code without suffering the intellectual properly loss dilemma
presented by the GPL. We have our work cut out for us and we need
work hard in the areas we are behind in order to remain relevant.
What are our challenges ahead? Our goals are divided broadly in three
categories.
The first category is technical:
- We need a better filesystem. We need a filesystem that will support
the ever growing disk sizes, does not need hours of fsck after a
crash, supports snapshots, replication and comes with a volume
management system that allows us to add and remove devices
dynamically. It would be nice for the filesystem to be able to
handle hardware faults gracefully by keeping multiple copies of
critical data, and being partially accessible.
- We need real-time scheduling support, POSIX real-time extensions,
and thread-safe libraries. There is an increasing number of
applications related to video, voice, and control that need hard
real-time support. We can start by bullet-proofing our thread
support and finishing the re-entrancy issues with our libraries,
then continue by evaluating real-time scheduling and making
subsystems of our kernel able to use multiple processors.
- We should modernize our network stack. Features such as SACK and ECN
are becoming necessary. We should support multiple default routes,
load balancing and throughput control. We need to fix our firewall
software immediately and in the longer run come up with something
cleaner and more extensible.
- We should improve our internationalization support.
- We need to replace our Loadable Kernel Module support with a real
kernel linker that will allow us to make the kernel more dynamic.
- We need better ACPI support that includes suspend and resume.
- As a longer term goal we should look into clustering, process
migration, and redundancy support.
- We should take advantage of pkgsrc to provide workstation or server
type systems with groups of software that can be easily installed on
top of the base installation semi-automatically.
I don't think that I am saying anything new here.
The second category is administrative:
- We need to produce more frequent releases. We must provide timely
response to security issues and have a better mechanism to supply
patches to supported releases.
- We need to institute better testing and try to provide more timely
feedback and concentrate on closing important bug reports. We should
also improve our installation process. While it is easy to install
NetBSD in our more popular platforms, it can be pretty daunting to
install on the more exotic ones.
- We need to make our committees more responsive and transparent. We
need to institute and document policies, protect our intellectual
property and trademarks, and get more people involved. We have too
few people in charge of many different positions and we constantly
fail to respond to the needs of the project.
- We need to improve our system services and administration support.
The third category is publicity, fund-raising, and the
project's image:
- We should setup fund-raising goals, and communicate what we are
going to be using the funds for.
- We should organize developer get-togethers and presence in major
conferences.
- We should contact the commercial users of our software and encourage
them to publicize their use of NetBSD as well as feedback fixes and
improvements to the code base.
- We should encourage more developers to join our project as well as
help them during their first steps, without intimidating and
criticize their work needlessly.
But we need your help to achieve all of that. And this help is not
limited in the technical areas. We need more people in the project
management and communications areas where traditionally we have shown
less focus.
Let's make this year different. Let's make the 3.0 release the best
one yet and on time. Let's make 2005 the year where YOU are going to
get more involved and make YOUR forward in the right direction.
Thank you.''
After Christos Zoulas' general overview, each presenter talked about the things their group achieved in the past year.
The overview of the Technical Executive Committee was given by Alistair Crooks:
``The Technical Executive Committee is in charge of four areas: the core team, release engineering, security-officer, and pkgsrc. Its current members are: Alistair Crooks, Lex Wennmacher, and Luke Mewburn. [...] Over the last year, there have been a number of achievements made by the various technical teams which come into the technical executive committee's area. I will let the various PMCs talk a bit about what has been achieved, any pressing issues or concerns which they have, and a bit about how they see the future over the next year. They have all worked very hard throughout the year in their own individual areas of expertise, and I'd like to commend and thank each team for their work.''
The core team presentation was given by Allen Briggs:
``Current core members are the same as last year:
briggs, christos, fvdl, lukem, matt
Random note:
* Core works on a rotation basis and attempts to get a majority
opinion for official rulings. This works pretty well most of the
time, but can be slow to collect responses from everyone.
* [...]
Some technical highlights, many since the 2.0 branch:
* statvfs(2)
* pf
* Major ipf updates
* New ports
* Many software updates (postfix, bind, groff, etc.)
* Many manpage improvements
* Many thread improvements
* Many audio improvements
* Many port improvements
* Version numbering changed with 2.0 release
* ptyfs
* PAM coming in now
2.0 goals that we missed:
* GNATS scrub. We still have a lot of outstanding PRs. Some folks
have done a great job in cleaning up some old ones, but we really
need to find a good way to do better as a project. Suggestions
welcome.
* Libraries all thread-safe. There has been a pretty good
improvement here, but there's almost certainly more work to be
done as was so tactfully pointed out recently on the mailing
lists.
Core concerns:
* Outstanding PR list. More than 12% of the PRs in our database
are still 'open.' We'd like to see that significantly down by
next year.
* Ports with inactive port-masters. There are some ports that have
been more or less abandoned. These should probably be picked up
or mothballed. If you love a port, please nurture it.
* Better organization of project roadmap. We're just getting
started with a roadmap. It would be nice to extend and subdivide
it to cover individual ports as well as different kernel and
userland subsystems. This might be a way to help re-engage some
port-masters.
* Testing. It would be nice if we can develop a lab or set of
volunteers to do some automated or at least regular testing for
functionality and performance.
* Long release cycles. The 2.0 release cycle was too long. We'd
like to find ways to work with releng to help reduce the release
cycle time in the future.
In addition to trying to address these concerns, we're looking toward
the future. The roadmap shows a number of things that we'd like to
see by 3.0, currently scheduled to branch at the end of February -- if
you've offered to do these things, please try to get them done or 3.0
will happen without them (I include myself in that admonishment):
* Implementing get*ent_r() and getXbyY_r()
* Flesh out wedges some more and start taking advantage of them
* Better ieee1394 support
* Updated GDB
* Modular nsswitch
* Complete PAM integration
* Better internationalization
* TCP/SACK
* EtherIP
* Merge of the ktrace-lwp branch
* POSIX shared semaphores
* XFree86 4.5
Looking further ahead, our roadmap includes:
* NFS4
* System packages
* Modular scheduler
* Updated GCC (4.x branch, if it's stable)
* Generic device hot-plug
* Better kernel locking primitives ("newlock")
* Migration to X.org from XFree86
* TCP/ECN
* Better support for modern parallel ports
* AIO support
Of course, these items and those suggested by Christos earlier are
only possible with the good work of all of the developers. Thank you
for your efforts throughout the years. It's a pleasure working with
you all, and I look forward to seeing what NetBSD can become!''
The security officer presentation was given by Dan Carosone:
``In previous years, Security Officer has provided details of the
following matters in our presentations. For more detail on these, see
the AGM logs [...]; today we'll provide a quick update, for those
entries which are likely to change over time.
* What does the Security Officer team do?
* Role of Security Officer team
* Responsiveness
* Projects
* Who is Security Officer?
(dan, david, groo, itojun & perry)
The security-officer role is intended to be primarily a coordination
and management function, providing a public contact point for NetBSD
security matters, and marshaling the project's resources, members and
activities towards various security-related goals.
There are two major aspects to the workload: incident-driven and
development-driven. This past year has, happily, been a relatively
quiet year for NetBSD security incidents (less so for issues that
affect pkgsrc). Unfortunately it's also been somewhat quiet for some
of our development goals as well. We laid out a number of goals at
last year's meeting. We've made mixed progress on these during the
year. For each of these, we'd like to give a self-appraisal and
summary of progress last year.
Last year's goals
-----------------
Goal: Continue to improve Binary Patches for Advisories
Appraisal: A
Summary: Binary patches issued for most Advisories
(where applicable).
* More automation to be done yet.
* Thanks to Releng for access to autobuilds
Goal: Re-organize Advisory publishing process
* Make CVS handling cleaner
* Lessen publication workload on S-O
Appraisal: B
Summary: CVS handling now seems to work well.
Automation of mail-out process helps greatly.
* Some 'toolchain' and familiarisation issues
with updating htdocs (more details below)
Goal: Get >75% of active developers to have PGP keys
* Helps for sharing confidential information
with relevant expert developers
* Increases security awareness and tool
familiarity among developers
* Increases visibility of NetBSD among security
savvy non-developers
Appraisal: B+
Summary: 104 developers' PGP keys
(in localsrc/security/publickeys/developers)
* Good degree of connectivity in web of trust
* All new developers will have PGP keys
* Has been useful in increasing the trust for
admin requests (eg, resetting ssh access)
* Not 75% yet, but getting closer; conferences
and drinking events seem to help and more are
encouraged :)
Goal: Security-Officer web page updates
* Help people understand who S-O is, does
Appraisal: F
Summary: Web Pages not significantly updated
* See 'staffing' below
Goal: Improve mail response time, Advisory publishing time
Appraisal: A-
Summary: Response to first contact email mostly quite fast
* Dropped the ball on very few issues - ( 1? )
* See 'staffing' below
* Fewer Advisories this year, fortunately
Goal: Improve internal S-O tracking and communication
Appraisal: B
Summary: Internal communication has become very good
No tracking system yet; delayed by a number of
infrastructure issues; working with admins now
and hope to start testing a new one shortly
Goal: Split Security-Announce from NetBSD-Announce
Appraisal: F
Summary: Security-Announce list created, but not cut-over
* Remains outstanding
* Questions about process and user preferences
* Was a lower priority, given available staff
time and workload
Goal: Better communication with Releng
Appraisal: B+
Summary: Regular exchanges between Releng and S-O via mail
and chat provided better coordinated response.
The one thing guaranteed to bring up security
issues is an imminent release.
Goal: Sign and Publish ssh host keys and PGP keys
Appraisal: B-
Summary: key tables were created, and made available via ftp
* Web pages not updated yet
* Coordinate with admins to keep host keys updated
Goal: Sign Releases
Appraisal: A
Summary: 2.0 release included a signed set of file hashes
* SHA1 and MD5, signed by s-o key
* Need to automate creation and maintenance
Goal: Establish and publish "Security Notes"
(quick 'NetBSD is not affected' announcements)
Appraisal: A-
Summary: The Security Note format was used, and well received.
* Need to update web pages to index them
Goal: Extended team of security volunteers, to help handle
non-confidential issues, and contribute time to
handle less critical flaws which are public and
not yet SA'd and to follow up with CERT and
provide NetBSD references for older CERT issues
Appraisal: F
Summary: We received only one inquiry from a volunteer this
year offering to help with ongoing S-O work
* We did solicit help on specific issues from
specific experts and groups like port-masters
* We didn't make a general plea to developers
This Year's goals
-----------------
* New Tracking tools
* Update web pages
* Improve updates to host keys files, with admins help
* Improve, extend Security Notes
* Extended team of security volunteers
Web Site updates
----------------
The www team has been completing the process of converting the NetBSD
website to XML. Making updates now needs a number of new tools
installed, and a number of different steps - which have changed as the
site develops. Overall, this is a hugely positive change, but at
least some of us have had 'awkward surprises' when trying to update
the website to publish a new Security Advisory.
Regardless of the internal format, there are considerable updates
required to the Security/ web page static contents, updating and
better documenting NetBSD's security presentation to the public.
Although we hope to be more successful with that work this year,
please feel free to make your own contributions.
For some time, we've also had a desire to improve the organisation and
presentation of the more data-driven aspects, such as automated
indexes linking SA's to affected releases (and, later, syspkgs).
There are a number of aspects to this we haven't had time to work
on. In the meantime, the VuXML project has developed a number of
useful formats and tools we want to look at carefully.
Staffing
--------
Like other developers, all the security-officer team members have
non-NetBSD work duties - 'day jobs'. This year, more than previously,
availability of time has affected S-O. For most of the year, although
quick incidents and issues were dealt with easily, few were able to
contribute significant time to 'heavy lifting' incidents or projects.
Therefore it was fortunate that there were fewer issues that affected
NetBSD itself this past year. Many of the issues notified to
security-officer were either not relevant to NetBSD, or were problems
in third-party code that affected pkgsrc. The pkgsrc team did a great
job in responding to those, and publishing the issues in the
audit-packages mechanism; they have our sincere thanks.
Security-officer always relies heavily on other NetBSD developers for
the actual resolution of many issues. Several times this year,
developers have found an issue in the course of other work, and
notified security-officer complete with a draft Security
Advisory. This is wonderful, it makes our job much easier, and greatly
improves the speed at which we can publish. Thank you, again, and
please keep it up!
Even so, given other commitments, we can always use more help. There
are several ways you can help listed below.
What can you do to help S-O?
----------------------------
* If you have the time, contact us about helping with the extended
team. It will have more relaxed urgency, and will help free up
S-Os time (and prevent burn out) for handling critical issues.
* Suggest and perform security sweeps for your pet issues.
* Review PRs in security category - and close them!
* Generate example systrace policies for system daemons.
* Write rc tweaks to run more daemons unpriv'd/chroot/etc.
* Stomp out more suid programs.
* Volunteer to join S-O! Some of us are, or will eventually be,
looking to retire and make room for new members.''
The release engineering presentation was given by James Chacon:
``Currently, the releng team consists of the following developers (in
alphabetical order of username):
- agc Alistair G. Crooks
- cjs Curt Sampson
- cyber Erik Berls
- grant Grant Beattie
- he Havard Eidnes
- itojun Jun-ichiro itojun Hagino
- jdc Julian Coleman
- jmc James Chacon
- jwise Jim Wise
- lukem Luke Mewburn
- msaitoh SAITOH Masanobu
- tron Matthias Scheler
Before I get started too much into things I'd like to make sure and
give a public thank you to everyone who worked on the 2.0 release. As
I'll detail below this was a very long process for us and the team
deserves many thanks for getting it done.
Now onto the specifics.
Major achievements last year:
- We got 1.6.2 out the door in March. While this was mostly a
maintenance release it did also address a number of security
advisories as well as numerous kernel fixes.
- We finally (after almost 8.5 months) released 2.0 to the
public. I'll go into more detail below as to what we hope to do in
the future to prevent these types of delays.
- We'd like to thank everyone involved in the src/x11 cross-over X11
build support. I won't try and name people specifically here
because I know I'll miss folks. The main thing is to highlight
just how much easier this makes the release engineering process as
everything that we put into a release directory on ftp.NetBSD.org
is now able to be done from one source tree.
- Maintenance continued on both the 1.5 and 1.6 source trees as
pullups were submitted and processed. (though I will note the
number of 1.5 submissions has been trickling off for some time).
- Increased communication with the security-officer team. This has
been problematic in the past as often times security issues pick
the worst timing to show up (as we're releasing is often a good
time it seems..), so keeping our schedules more coordinated is
helping both teams out here.
- Better communication with the rest of developers via requested
semi regular status reports during a release cycle.
Plans for this year:
- Start the release of a security/critical fix branch from each main
release. As detailed recently to netbsd-developers this sub-branch
will consist of security/critical fixes only and is expected to
get us binary release updates that security officer also needs
when publishing an SA. It's expected that 2.0.1 will release soon
(as the new autobuild hardware is installed and sysinst support
for updates is added into the tree).
- Release 2.1 sometime during spring of 2005
- The return of regular snapshot builds once the new autobuild
hardware is in place.
- Branch 3.0 at the end of February. NOTE: this doesn't mean we plan
on overlapping 2.1 and 3.0. It's expected that 3.0 won't be ready
until probably June so at least a month or more will exist between
these releases.
- Release 1.6.3 during 2005. It's unclear at this time when a good
point in time for that will be, but as it may be the final release
of the 1.6 tree it's release most likely will happen shortly after
3.0 releases and the 1.6 branch is closed down.
- De-support/close down the 1.5 branch. There are no pullups
currently waiting on this branch and an announcement about it's
status should be going out shortly.
Major hiccups/problems during 2004:
- Obviously to most of us 2.0 took way too long to get into a
releasable form. There were a large number of factors that led to
this but the main thing that hamstrung us early on was getting ipf
into a stable state that was ready for release. While it was
critical to get onto the ipf 4 code in the future the initial
branch should be pushed out if a major import like this happens at
the last minute.
- As everyone is aware this is a volunteer project so people have
"real life" to worry about too. What this can mean when we drag a
release out is conflicts arise with real life vs project
schedules. So doing anything we can to keep things moving and on
schedule helps everyone avoid stress :)
- In addition we lost both releng machines during 2004. The main
machine we published snapshots and did request tracking on
(releng) died and during the build cycle for 2.0 it was found that
the autobuild machine (tgm) was having memory timing issues. As a
result of this new hardware was purchased and will be installed
shortly.
Things to improve on we need help with:
- Portmaster involvement. We need the portmasters to be more
involved with the functioning of their ports. This includes
testing out the branch in enough time before the release to ensure
there are no MD issues
- Testing. More testing and feedback is desirable. Especially the
case with pullups during the release cycles.
- Bug fixing. As part of the 2.0 release we tracked a given set of
PR's highlighted as "critical" to fix before release. Yet, some of
those would sit for months before getting worked. This puts
release engineering in a bind as we don't want to release code
with known critical bugs, but without people stepping up to work
bugs it makes things drag out for a long long time.
- More manpower never hurts. Especially as we try and ramp up the
critical/security branches for release having additional people
willing to commit the time to coordinate releases will be crucial.''
The pkgsrc presentation was intended to be given by Johnny Lam, but, unfortunately, he couldn't make it. Thomas Klausner gave the presentation for him:
``The pkgsrc PMC is the Project Management Committee which
oversees the development of pkgsrc, the NetBSD Packages
Collection. The pkgsrc PMC currently consists of Alistair
Crooks, Thomas Klausner, and Johnny Lam (agc, wiz, jlam).
Progress made in 2004
=====================
pkgsrc continues to grow in the number of packages supported
in the tree, and in the number of operating systems to which
pkgsrc has been ported. Particularly noteworthy progress has
been made in the following:
* Pkgsrc documentation converted to XML and placed in
pkgsrc/doc/guide. This replaces the older monolithic
Packages.txt, and we can now generate pkgsrc documentation
in different formats, e.g. HTML, PDF, etc. [grant, jschauma,
hubertf]
* Full switchover to buildlink3 and the removal of all buildlink2
code from pkgsrc. This makes our infrastructure cleaner
and eases pkgsrc development. [wiz, snj, minskim, xtraeme,
and many other developers]
* Wrapper scripts broken out of buildlink3 into a standalone
framework. This was an important step toward making it
easier to port pkgsrc to other operating systems and to
older releases of NetBSD. [jlam]
* Ports of pkgsrc to additional operating systems:
- Interix (Microsoft Services For UNIX) [tv]
- DragonFlyBSD [Todd Willey from DragonFlyBSD Project]
* bootstrap-pkgsrc was merged into pkgsrc, which simplifies
the task of getting pkgsrc running from scratch on any of
the operating systems supported by pkgsrc.
* A pkgsrc regression suite was added into pkgsrc/regress to
improve our quality control for commits to pkgsrc. [gavan]
* A build-time options framework has been added to pkgsrc to
more uniformly describe the optional components that may be
built into packages. [jlam]
* Better execution of branching process -- we stuck to our
goals of have well-defined limits to pkgsrc "freeze" periods,
and we have managed to branch regularly at the end of each
quarter for the past 4 quarters. [pkgsrc-pmc]
The following items are part of a good trend -- the involvement
of more pkgsrc developers in helping to maintain pkgsrc instead
of simply committing new packages. More developers were given the
freedom to take on new responsibilities:
* Improved management of the pkgsrc-stable branch with the
establishment of a releng-pkgsrc team. This team uses the
usual releng tools to manage pullups to the pkgsrc-stable
branch in a timely manner. [admins, releng-pkgsrc]
* Much more responsive pkg PR handling thanks to a rotating
team of developers (modeled on www@) that routes incoming
PRs to other developers most likely to be able to handle
them. [pkg-bug-handlers]
* Dedicated groups of developers using pkgsrc on non-NetBSD
platforms handle platform-specific PRs. This helps ensure
that pkgsrc support for these platforms continues to improve
over time. [solaris-pkg-people, darwin-pkg-people,
linux-pkg-people, irix-pkg-people, freebsd-pkg-people,
interix-pkg-people]
* Continuous bulk building of pkgsrc on several different
platforms to expose broken packages or broken infrastructure
in a timely manner. [krister, agc, minskim, jschauma, and
many others]. Thanks go to wiz for providing diffs of the
bulk build results.
Work in Progress
================
* tv-recurse branch has changes to pkgsrc infrastructure make
calls so that we aren't as reliant on recursive make
invocations. This will allow for substantial speedups
when building large numbers of packages. [tv]
* Package views is still considered experimental pending a
solution to the problem of shared directories across
packages. Two different ways of solving this problem were
discussed in November 2004 on the tech-pkg mailing list,
and both solutions have developers (jmmv, jlam) committed
to implementing them during the course of 2005.
Other noteworthy items
======================
* Thanks go to jmmv and markd for their respective high-quality
jobs in updating the GNOME-2.x and KDE-3.x packages in pkgsrc
to the latest versions. These are huge collections of highly
visible packages, and they have good maintainers.
* Sun has donated hardware for pkgsrc development.
* Sun will be using "some form of pkgsrc" for its contrib
packages extras in Solaris 10.
* pkgsrcCon 2004, hosted by wiz & dillo at TU Wien, was deemed a
success. pkgsrc developers and users were able to get
together and share ideas/concerns/projects over a three-day
period. pkgsrcCon 2005 will be hosted by salo in Prague on
May 6-8 to hopefully start a tradition of being a well-spring
of pkgsrc projects for the coming year.
* DragonFlyBSD has some strong advocates within the project
for using pkgsrc as its package system instead of FreeBSD
Ports or rolling their own. This is a tribute to the
excellent job that pkgsrc developers are doing to produce
a very good and very usable package system, and an
acknowledgment of how well we support non-NetBSD systems.
Plans for the future
====================
* Continue to work on using cross-compilers to help build
pkgsrc for slower machines on much faster ones. reinoud
has posted a solution using distcc to tech-pkg@, and kristerw
presented interesting work at pkgsrcCon 2004 that will
hopefully be integrated into pkgsrc in the future.
* Decrease our bulk build times by using clusters to spread
the load. jlam will be presenting his work on this at
pkgsrcCon 2005.
* Be more responsive about security problems, perhaps by
creating a group similar to security-officer for pkgsrc.
* Improve bulk build results on non-i386 architectures. The
two items above will help this greatly.
* Usual stuff:
- Create more packages!
- Maintain your packages!
- Port to more platforms!
- Attract more pkgsrc developers!
Where you can help
==================
* If you have knowledge of how a particular feature works in
pkgsrc, or if you add a new feature to pkgsrc, please try
to write a regression test to test the proper functioning of
that feature. This will help keep pkgsrc working as expected
by allowing developers to test that their commits don't fail
the regression tests.
* Consider joining any of the various teams that help out with
handling PRs, writing user documentation, or managing the
pkgsrc-stable branch.
* pkgsrc-wip continues to grow and to attract new
developers-in-training. It would be very helpful to the
pkgsrc project if more developers could find time to look
over requests for package reviews on the pkgsrc-wip mailing
lists. This helps to spread the knowledge, and to gain more
people capable of answering pkgsrc developer FAQs for new
pkgsrc-wip developers and users.''
The report of the Communications Executive committee was given by Tracy Di Marco White:
``The Executive Committee for Communications Members:
Tracy Di Marco White <gendalia>,
Hubert Feyrer <hubertf>, vice chair,
Jan Schaumann <jschauma>,
Luke Mewburn <lukem>, chair,
Perry E. Metzger <perry>,
Jeremy C. Reed <reed>,
Scott Reynolds <scottr>
Review of the past year:
* A new logo was selected out of over 400 submissions by
238 artists. IP transfer was handled by Luke, and
now TNF owns its new logo.
* We've been handling various announcements, coordinating
with release engineering for the 2.0 release
announcement, the trademarking for NetBSD, etc.
* Jan has done a marvelous job coordinating the quarterly
reports, advertising what we're doing so that people
are made aware of our developments.
* Making sure various news sites such as slashdot,
daemonnews, osnews are made aware of NetBSD related news
by submitting news items to them.
* Put together an org chart of the TNF structure.
http://www.NetBSD.org/foundation/structure.png
* Hubert worked with the German vendor JF Lehmanns
(www.lob.de) to have CDs available in Germany and
Europe.
* Coordinating, helping, and performing presence of the
NetBSD project at various tradeshows, and advertising
them on http://www.NetBSD.org/gallery/events.html
* Hubert has acted as point of contact for the benchmark
Gregory McGarry did, measuring performance of FreeBSD
5.3 and NetBSD 2.0 providing webspace
(http://www.feyrer.de/NetBSD/gmcgarry/), and making the
article public for Gregory.
* Hubert followed up discussion on NetBSD benchmarks by
posting a list of comparisons of NetBSD to other
operating systems, and posting the list to
tech-perform@, see
http://mail-index.NetBSD.org/tech-perform/2005/01/12/0000.html
* Jeremy has converted the logo into a format cafepress is
happy with, and a NetBSD t-shirt is available. The
price will be set to generate funds for TNF.
http://www.cafepress.com/netbsd
Issues we'd like improvement on:
* Support for active marketing. Hubert wanted to print
official NetBSD stickers for distribution, but the
budget wasn't available.
* Better participation and flow of information from
developers: documenting things properly (src/doc/CHANGES,
htdocs) so our users can find documentation about things;
* The cafepress free store only allows one item of each
type. We'd like to be able to have more.
Goals:
* Fundraising Fundraising Fundraising
* Establish ourselves as a point of contact for announcing
"interesting" things, taking data from developers (see
"participation and flow of information from developers"
above) and preparing information to hand over to the
www@ team, the netbsd-news@ and/or netbsd-announce@ list
as well as various news sites (slashdot, daemonnews,
OSnews, ...)
* Hubert has established a personal blog with NetBSD
related news items that were not achieved by TNF but
that are still interesting to NetBSD users/fans.
Advertising these to the NetBSD community and beyond is
good public relations.
* Several people are starting to do various tradeshows.
Discuss organization steps to help each other, and as
comm-exec@ contact t-shirt and CDROM vendors to donate
items for handing out at these events.
* Calendar with trade shows and events for upcoming 12
months and find developers or volunteers to begin
registration and planning for events well in advance.
(For example, LinuxFest Northwest is coming up in April.)
* Build media contacts list for mainstream media: Dr.
Dobb's, InfoWorld, Network World, etc. (They have covered
BSD news a few times each year.)
* Media relations schedule: send out at least one press
release each month to various mainstream tech media (not
just small BSD-specific sites).
* Work better with release engineering to know when a
release is out. Mainstream press needs lead time to be
able to announce our releases.
* Work with developers to co-write technical articles for
technical magazines and technical events. Build a list
of articles that should be done, target (magazine or
event), and developers to help.
* Continue advertising NetBSD and support people who do so.
* Fundraising Fundraising Fundraising
Thank you, and please consider helping promote NetBSD and
pkgsrc.''
The www team presentation was given by Jan Schaumann:
``As in previous years, let's start out with who the www-team
currently is. While there have been the same regular members on and
off again, at the moment the rotation consists of:
* D'Arcy J.M. Cain <darcy>
* Klaus Heinz <heinz>
* Jan Schaumann <jschauma>
* Jeremy C. Reed <reed>
* Simas Mockevicious <symka>
Daniel de Kok <daniel>, who has been a very active and helpful member
of the rotation, regrettably had to drop out recently due to other
commitments.
These members take turns in answering the various questions that are
sent to www@ and perform the regular maintenance work. More formally,
the duties of the www team are outlined at
http://www.NetBSD.org/people/groups/#www and
http://www.NetBSD.org/developers/www.html.
To summarize, we're somewhere in between of ``customer-service'' /
``tech-support'' and HTML-monkeys. Aside from that, the www team also
coordinates with communication-exec and the general developer body to
prepare official announcements not only on our own website, but on
other forums as well.
The www team also responds to mail coming in on the mirrors@ address
and maintains the list of our various volunteer mirrors in
coordination with the admins team.
So what have we been up to over the last 12 months? In preparation
for last year's annual meeting, we had set out a number of goals, and
while we have tried to work on these goals, we were not able to
accomplish all of them:
* translation work has stagnated in some areas / languages but also
improved in others. The Korean and Lithuanian translations
(maintained by minskim@ and symka@) are currently the most active
translations efforts.
* news announcements and general advocacy through our website has, I
believe, improved somewhat, and cooperation with other committees is
smoother now as well.
* we did *not* manage to get any work done at all on
projects.NetBSD.org. As was pointed out last year, this server
contains a large number of completely dormant projects. These
should be updated.
* developers' public keys have *not* yet been made public on
the website. This is an item that has completely dropped
off the radar and we probably should file a PR for this to
remind us.
So, you may ask, have we done nothing useful at all in 2004?
Well, we did achieve a few things over the last 12 months, and I feel
that we are still continuously improving:
It is worth noting that the NetBSD guide (basically a project all on
its own) has been revamped and updated for the 2.0 release. The
overwhelming majority of the work in this area has been done by Hubert
Feyrer, who we owe much thanks for this.
We have published the new logo together with communication-exec.
While this of course lead to a lot of discussion, the process of
updating the entire website and virtually every single page has shown
us weaknesses in our infrastructure. At the moment, there are 2244
HTML files in the htdocs repository. Of these, 299 are generated from
XML (178 at the time of the publication of the logo) and 396 from
.list format (via perl). So for any global change, we still need to
hand-edit 1549 files. Yuck!
This is why we have focused in the last quarter of 2004 on the
conversion of the existing pages to XML. While this brings its own
difficulties, it has a number of convincing advantages:
* all pages generated from XML have the same consistent look and feel
* if the need arises, we can easily change the look and feel of the
entire website
* common segments (header, footer, styles etc.) can be controlled in
one central file, so that a simple 'make' in the root can regenerate
all files
* all files are automatically standards compliant
Some of the problems caused by the use of this framework are the large
number of packages that each developer needs to have installed.
Another problem is that different versions of the required packages
produce different output, leading to a number of unnecessary 'regen'
commits. This has lead to some ``awkward surprises'' and frustration
among developers who have to touch htdocs occasionally.
We are understanding of this frustration, recognize the shortcoming
and have started to work on a remedy for this situation: We have
started to change our point of view of the files under CVS to accept
that all files that are generated from another source, should be
treated as ``object files''. This means that they should not really
remain under CVS control -- instead, developers should just commit the
source files and be done with it.
I'm currently working on setting up www.NetBSD.org such that it can
build the htdocs tree itself. This setup is virtually complete and
currently in a testing phase. Once we're satisfied that everything
works as planned, we can remove the .html files from CVS. This has
the advantage that it doesn't matter anymore which version of which
required package a developer has installed. In fact, developers who
routinely do not do htdocs work can just commit trivial changes to the
source files without the need to generate the html files themselves.
The setup on www.NetBSD.org also acts as an implicit error-checker: if
a commit breaks the generation of the website, it will complain and
the website will _not_ be updated.
So, to summarize (and reiterate), while we understand the frustration
with the difficulty of using the XML framework, we hope that the
developer community understands that in the long run it will make
things Much Easier. And while we did not achieve many of the
individual goals set out last year, I nevertheless believe that we
accomplished _some_thing and had a somewhat successful year.
General goals for the coming year, goals the achievement of which each
developer is encouraged to contribute to, are:
* remove redundant documentation and eliminate duplication: There are
many pages that contain similar information. They should be merged
with other documents, moved into the guide or purged altogether.
* continue to convert files to XML
* revive / continue with translation work: Each developer who is even
semi-fluent in any languages other than English is strongly
encouraged to help with this task.
* create a framework to manage ports-pages wrt to new: releases and
news items: the 2.0 Release pointed out that at the moment there is
no simple way of updating the largely identical information on ~60
pages.
* entice port-masters to keep their port-pages up to date: Many of the
port-pages have not been updated in *years*. Surely there has been
some progress or some noteworthy achievement, yet the www team can
not keep track of all the changes in all ports.
* consider a redesign of the main page: The introduction of the new
logo has lead to some discussion about the general design of the
main page. While many users like the simplicity of the layout, it
has become clear that none of us is a web designer. Several users
(and developers) have made suggestions as to how to improve the
websites layout, but the danger of (yet another) huge bikeshed
discussion has inhibited any progress. Which actually is too bad.
And with these suggestions for discussion and goals, I'd like to close
the presentation of the www-team. I'd like to thank all the members
on the www team for their hard work and good spirits.''
Lex Wennmacher presented the report from the finance committee:
``The duties of finance-exec are managing the financial business of the Foundation. The current members of finance-exec are Christos Zoulas (chair), Alistair Crooks (vice chair), Lex Wennmacher (treasurer), and Andrew Brown (assistant treasurer). Our annual expenditure is mainly on servers, disks, and miscellaneous hardware, but also on legal fees. Currently all of our expenditures have to be balanced by our only income, donations. Since the last Annual General Meeting, we received about 65 donations, some were quite generous. We acknowledge donations on our web pages, if the donor so wishes. The release announcement for 2.0 contained a solicitation for donations which was remarkably successful, as we noticed a significant donation increase. This concludes finance-exec's presentation.''
The presentation of membership executive committee was given by Thomas Klausner:
``Membership-exec oversees all aspects of membership in the
Foundation. The current members of membership-exec are Lex Wennmacher
(chair), Christos Zoulas (vice chair), Tracy Di Marco White, Thomas
Klausner, and Ken Hornstein.
The routine work of membership-exec consists of handling and deciding
on new applications. Since the last Annual General Meeting, we have
received 17 applications, had 12 new developers, and 2 resignations.
We lost one developer, Eric Reid, in a tragic accident.
One of the principal duties of membership-exec is keeping the records
relating to the membership of the Foundation, used for example when
preparing the list of Active Members for determining who may vote in
Board Elections. Our bylaws require that developers must have signed
a Membership Agreement before they can be considered members.
Another achievement in 2004 was the finalization of an improved
Membership Agreement - the old version of the agreement didn't
properly reflect the new structure of the Project and was legally
problematic in several ways.
For this year we've got the following items on our agenda:
- implementing a developer contact database (containing
postal addresses and phone numbers)
- developing a procedure for validating developers when
they lose their ssh key
- developing a procedure for resignations
That's all, thanks for reading.''
The presentation of the Executive Committee for Administration and the Project System Administration group was given by Tracy Di Marco White:
``Members:
Christos Zoulas <christos>, on call, AEC vice chair,
Tracy Di Marco White <gendalia>, on call, AEC chair,
Bill Squier <groo>,
Jan Schaumann <jschauma>,
Phil Nelson <phil>, www synchronisation,
Scott Reynolds <scottr>, AEC,
Jonathan Perkin <sketch>, on call
Noriyuki Soda <soda>, on call
Soren S. Jorvang <soren>, on call
S.P.Zeidler <spz>, on call
Jason Thorpe <thorpej>,
Thor Lancelot Simon <tls>
New hardware:
* A new backup server locate at Ludd, University of Lulea, Sweden,
maintained by Anders Magnusson has been set up, and Thor has
just started doing backups of everything we can fit onto it.
* New http, console, mail, and one xen box that is used for
administering the other Project servers. Thor built the system
disks, and Erik Berls & Eric Volpe assisted in the installation
of our new machines, as well as help from Peter Losher of ISC.
* Two new build boxes were put together by Thor, and are currently
on their way to California for Jason to install.
* Sun has donated two machines for use in doing Solaris pkgsrc bulk
builds, hosted at Stevens Institute of Technology, where Jan
Schaumann maintains them.
Hardware failures:
This has been a bad year for us, at least regarding our hardware.
We'd like to thank everyone for their patience during our outages.
* The drives failed on the new http machine shortly after it went
into service, causing me to do a fairly quick rebuild onto the new
mail server hardware. Not long after that, the drives in the NEW
new http server also showed some errors. Thor ordered new raptor
SATA drives to replace these drives, and Jason put them into
service. With the new drives in place, we should be able to move
the mail server to the new hardware in the near future. While
www.NetBSD.org was unavailable, Manuel Bouyer made
www.fr.NetBSD.org available to answer www.NetBSD.org. Soda
recovered mail lost because mail-index was unavailable, and Soren
refed everything into mail-index.
* The release engineering tender machine (releng) had a power supply
/ motherboard failure. Todd Whitesel picked up the machine to
investigate the problems and how to repair it, but was unable to
make it boot. I moved the release engineering ticket tracking
service (req) to www.NetBSD.org, with help from Jim Wise, after
Erik Berls mounted the hard drives and put the data up for me to
use to reinstall it.
* The release engineering build machine failed to successfully build
various 2.0 builds due to various hardware failures. We were
unable to use it for the 2.0 release sets.
* The ftp server's raid controller failed multiple drives, causing
Thor & I several days of pain while we backed up what we could and
rebuilt and reinstalled it. Peter Losher and Eric Volpe were
immense help juggling disks and installing new disks. The
multiple disks failures were traced to a bug in the raid
controller firmware. Many people were involved in this, if I've
forgotten anyone I apologize.
* The ftp server suffered from some frequent crashes that were
traced to memory errors, solved by reseating the memory, and
upgrading the bios.
* The cvs server failed a disk, but after testing we determined the
disk was actually ok. The machine was able to rebuild onto a hot
spare for the duration.
* The anoncvs server suffered a raid controller failure that
manifested as single bit errors occurring frequently in our
publically available source trees. After significant
investigation, we traced this to the raid controller, and Thor
provided a replacement raid controller. Petra, our newest admin,
had this fail in her first on call shift, and worked quickly to
make available update from the ftp server that are normally only
available from the anoncvs server.
For all of our hardware failures, Peter Losher of ISC provided
invaluable help, swapping hardware and escorting our helpers
in to do hands on work.
Changes:
* We have simplified and corrected the trust relationships between
the Project servers. [...]
* We have begun to migrate towards a standard hardware and software
package for Project servers, replacing many old 4U machines with
non-redundant disks and random old versions of NetBSD with modern
1/2U servers running 2.0. We have decreased our rack usage by
more than 10U. Thor has done the majority of the design work
involved, and Jonathan has started working on a system to ease
package rebuilds for all of our Project servers.
* We have established an on-call system for handling day to day
admin tasks, and added new admins to the on call rotation. We are
working to make sure issues brought to our attention don't fall
through the cracks.
* Soren Jorvang performed our migration from qmail to postfix, with
some help from scripts Christoph Badura wrote after studying our
qmail/majordomo configuration.
* Soren also performed the migration from gnats3 to gnats4, and in
the process also moved gnats from the mail server to the web
server, allowing real time access to the gnats database from the
web.
* I implemented greylisting (light) with blacklisting under qmail
this year. This caused some delay, but was widely appreciated
given the amount of spam everyone was receiving through our mail
host. On moving to postfix, Soren implemented greylisting using
gld, which has caused much more comment, but has done an even
better job of making our mailing list owners lives reasonable.
We're currently testing a lighter version of gld.
News:
* We've established our own contract with 200 Paul
(www.e200paul.com), so we can have our own contact able to
physically work with the ISC hosted systems without being escorted
by someone from ISC. Jason Thorpe has agreed to be our hands, and
we deeply appreciate his ongoing help in resolving a problematic
access situation that has persisted for years. This also allows
us to directly ship hardware to 200 Paul.
* ISC provided us with a /27 network after we ran out of room in the
/28 we were previously using.
Upcoming:
* We will be implementing RT for use for security officer and for
admin tasks. We are currently discussing the best way to do this.
* Soren is looking into upgrading the mailing lists from majordomo
to majordomo2.
* Soren is also investigating moving mail-index from home grown
scripts to mhonarc to facilitate easier access to the mailing list
archives.
Goals:
* No broken machines!
* Complete development and deployment of new backup system for all
TNF hosts.
* No broken machines!
* Build on ongoing standardization of hardware and software
configurations by deploying centralized configuration management.
* No broken machines!
Thank you for your patience through all our changes.''
After all the reports, Christos Zoulas thanked all the presenters for the interesting information and insight into the various project groups. If you are interested in participating in ongoing discussion, please join our mailing lists and see our web site for more information about the NetBSD project, http://www.NetBSD.org/.
-- Hubert Feyrer <hubertf@NetBSD.org>,
2nd Chair of the Communications Executive Committee,
The NetBSD Foundation
Back to the NetBSD Foundation Inc. page