The NetBSD Foundation Press Release: NetBSD and the Google "Summer of Code" 2008 Summary

Implement Ext3 file system support

Student: Rus-Rebreanu Alin-Florin
Mentor: Alistair Crooks, Dieter Baron

Summary and goal:  NetBSD currently provides access to files on Linux ext2fs file systems through its own implementation of the ext2fs interface. The ext3fs file system has extended the ext2fs file system abilities, mainly to provide journalling functionality. Building on, and learning from, Wasabi System's recently-contributed WAPBL functionality for BSD FFS, this project extends NetBSD's ext2fs file system to provide journalling functionality for ext3fs file system images.

Results:  After a slow start, initial contact was made, and various ISO images were provided to the student, and advice on kernel debugging was given. After that, contact was lost with the student. The project did not succeed, or even get past the initial contact stage. The overall impression from this project was that the student was unable to take advantage of the help provided for him, which resulted in the loss of a great opportunity.

Create an in-kernel API for "packet classes"

Student: Anish Babu
Mentor: Herb Peyerl

Summary and goal:  The project's goal was to create an in-kernel API for "Packet Classes" and for labeling packets with their classes for special treatment by traffic shapers and Network Interface drivers.

Results:  The project was not a success. The student failed to get an adequate start and was uncommunicative with his mentor.

File system access utilities

Student: Ysmal Arnaud
Mentor: Antti Kantee

Summary and goal:  The project's goal was to use rump and ukfs to write command line utilities to access file systems, much like mtools. This style of approach requires no kernel support to access and modify file system contents. All of the NetBSD kernel file systems supported by rump and ukfs are supported by the utilities. This means for example ffs, ext2fs, msdosfs and udf.

Results:  The project was a tremendous success. The following fsu-programs were implemented: cat, chflags, chmod, chown, cp, diff, du, exec, find, ln, ls, mkdir, mkfifo, mknod, mv, rm, rmdir, touch. These utilities share usage with the regular counterparts, but use ukfs for file system access. They are installed with the prefix "fsu_". As an example, fsu_ls /path/to/image -laFTrS somedir will give a directory listing like if /path/to/image was mounted and ls -laFTrS somedir executed from the mountpoint root directory. In most cases the utilities can autodetect the file system type, so there is no need to specify it. Additionally, the following programs were implemented:

fs-utils is available from pkgsrc from under filesystems/fs-utils. Arnaud has become a NetBSD developer (ie been granted commit privileges) and is working on integration with the base system. It will require refactoring the original utilities for code sharing purposes.

Expansion for wstablet in NetBSD

Student: Jason W. Beaudoin
Mentor: Petra Zeidler
Co-Mentor: Michael Lorenz

Summary and goal:  wscons is an abstraction layer between hardware drivers and userland, allowing userland programs to not need to know about specific drivers, nor when the wsmux feature is being used, if there is a physical device at all (or how many in total). Currently there is a driver for pointer devices included in wscons, wsmouse, that also will handle absolute coordinate pointer devices (like touchpads), but will not handle additional features of graphics tablets like pen pressure, pen tilt or pen identity. The mandatory goal of the project was to create a wscons driver for graphics tablets (also known as digitizers), which was to be based on the wsmouse driver but which should support all the additional tablet capabilities. Sample drivers both for hardware and for X were optional elements of the project.

Results:  While the student managed to just barely fulfil the mandatory requirements of the project, the results are not in a state to be included in NetBSD, nor to be useful to anybody yet.

Hurd translators

Student: Marek Dopiera
Mentor: Aymeric Vincent

Summary and goal:  The long term goal of this project is to add support for HURD translators into the NetBSD project. The goal of the SoC part of it was to develop a prototype implementation consisting of the necessary userland tools and ext2 filesystem modifications, the ability to launch translators on demand, and to start the GNUMach emulation layer.

Results:  The project is not yet ready to be integrated, and was too wide for a Google Summer of Code. Modifications to the filesystems and userland tools can be integrated into NetBSD's source tree, but the translator functionality needs much more discussion among the community. The GNUMach emulation work done during the SoC will be integrated and worked on. Finally, Marek Dopiera will probably become a NetBSD developer and be given commit privileges.

Write and improve NetBSD LVM driver

Student: Adam Hamsik
Mentor: Brett Lymn

Summary and goal:  The goal of this project was to add support for the LVM devices to the kernel, improve the userland/kernel communication and port the LVM userland tools to NetBSD. It was decided to limit the LVM device to a simple linear driver to keep the project to a reasonable scope. The Linux LVM subsystem uses a large IOCTL structure to pass information to the kernel drivers, the project goal was to change this IOCTL to a proplib interface but provide a compatibility library to allow easy porting of the Linux userland tools. The last goal of the project was to port the Linux userland LVM tools to NetBSD.

Results:  This project was a great success, Adam demonstrated a working linear LVM driver, a kernel interface converted to use proplib and ported the Linux tools to the NetBSD source tree. This work has been merged into the main NetBSD source tree.

Add support for UVC devices (USB web-cams)

Student: Patrick Mahoney
Mentor: Jared D. McNeill

Summary and goal:  The aim of this project was to develop a Video4Linux2-compatible API along with a reference driver for USB Video Class-compliant webcams. Also part of the deliverables were documentation in the form of man pages for the new user and kernel APIs, and porting of existing applications to use with the new API.

Results:  Both the Video4Linux2-compatible API and the UVC reference driver were imported into the NetBSD source tree and will be part of the upcoming 5.0 release. Documentation was provided for both the user and kernel APIs, and one other webcam driver has already been written and committed using the video device abstraction layer. The application porting was not completed; however, Patrick did write a test application and the Video4Linux2-compatible layer was written in such a way that it was trivial for NetBSD developers to port existing applications. The NetBSD package collection now contains a diverse set of applications that can speak to video capture devices, including but not limited to: Ekiga, MPlayer, Cheese (via GStreamer), UCView, and VideoLAN Client. Patrick has since become a NetBSD developer and been granted commit privileges.

DVB drivers and kernel framework

Student: Jeremy Morse
Mentor: Jared D. McNeill

Summary and goal:  The aim of this project was to develop a digital TV capture API compatible with LinuxTV along with a reference driver for the hardware of the student's choice.

Results:  Empia Technologies kindly donated hardware for this project and Jeremy received an em28xx-based USB DVB-T tuner. Jeremy began development by writing the digital TV capture API along with a pseudo-device driver that fed a test pattern as a transport stream through it. Once completed, he ran into an unanticipated roadblock; the NetBSD USB stack lacked support for isochronous transfers on hi-speed controllers. Jeremy ended up adding support for hi-speed isochronous transfers, which was quickly imported into the NetBSD tree before the project was completed as it was very useful for the UVC project as well. The final part of the project was to port Empia's em28xx driver, which was a success. After the project was completed, we decided that due to licensing headaches and the numerous combinations of hardware that is on the market, the approach for handling tuners and demodulators needs to be revised. One other digital TV capture driver has already been written using Jeremy's APIs, and a simplified rewrite of the Empia em28xx driver has also occurred. The digital TV capture API is expected to be included as part of the NetBSD 6.0 release, and time permitting as part of NetBSD 5.1 as well. Jeremy has since become a NetBSD developer and been granted commit privileges, and his hi-speed isochronous code has already been adopted by OpenBSD and FreeBSD.

pkgsrc: rewrite wrapper framework in C

Student: Amitai Schlair
Mentor: Johnny Lam

Summary and goal:  pkgsrc achieves most of its portability through subsystems integrated into its infrastructure. Among these subsystems is the "wrapper framework", a complex admixture of sh, awk, sed, and other tools intercept calls to the compiler and linker and perform various transformations before passing the calls through. The wrapper framework has worked reasonably well for some time, but its internals are difficult to understand and it is a known performance bottleneck. The object of this project was to eliminate this bottle neck by way of a reimplemented in C, following a test-first methodology.

Results:  The student was unable to start the project on time, and the project goals had to be revised early on. The student did deliver some of the revised milestones. However, in the end personal reasons prevented him from committing much time to the project, communications stalled and no progress was made, causing the project to fail in the end.

Improve syslogd

Student: Martin Schütte
Mentor: Christos Zoulas

Summary and goal:  The syslog daemon handles log messages from shell-scripts, applications, daemons, the kernel, or by network. It writes them into logfiles, on user's consoles or forwards them to some other logserver -- all depending on its configuration and the message properties. The project's goal was to implement the upcoming IETF standards for Syslog:

Results:  The project was successful. The improved syslogd is one of the first implementations of the new standards. Work has been merged into NetBSD-current and Martin has become a NetBSD developer and been granted commit privileges.

Converting remaining regression tests to the Automatic Testing Framework

Student: Lukasz Strzygowski
Mentor: Julio Manuel Merino Vidal

Summary and goal:  For a long time, NetBSD has had a small set of regression tests in the src/regress directory. These tests were ad-hoc utilities that checked for specific functionality, but the tests were not tied to each other in a consistent way. In Summer of Code 2007, NetBSD got a new testing framework known as the Automatic Testing Framework (ATF for short) with the major goal of providing consistency among the test programs and an easy way to execute them in an automated fashion. Along the process, only a small, representative subset of the old tests were converted to ATF, which were all added to the src/tests directory. The goal of the atfify SoC 2008 project was to convert all the remaining tests to the new framework, thus being able to drop the old code.

Results:  At the end of the SoC 2008 project, almost all of the old regression tests were converted to the new framework. 139 new test programs were implemented, which provide a total of 393 test cases. There is still some work left to do, but a lot of progress was made. Furthermore, some improvements were made to ATF itself to make it faster and to add more features to the shell interface; these new features are used extensively by the new tests. The deliverables of this project are almost ready to be imported into the NetBSD tree. The plan is to start by releasing ATF 0.6 and then to import the tests into the NetBSD tree.

Subfiles for FFS

Student: Adam Burkepile
Mentor: Phil Nelson

Summary and goal:  Many file systems have a form of subfiles, such as Alternate Data Streams in Window's NTFS and resource and data forks in Apple's HFS. This project was to add the basic support for subfiles to FFS on NetBSD. This is not a port of any other subfile implementation but the addition of the core elements to a subfile implementation. A complete implementation was not part of the plan.

Results:  The results of this project was a definition of the basic subfile interface and implementation of subfiles for FFS. Changes to the superblock, mkfs(8), and dumpfs(8) along with the UFS routines provide the initial basic support including the creation of subfile directories. This project did not make it as far as projected and did not get a working subfile implementation.

Implementation of RFC4380 (Teredo)

Student: Arnaud Lacombe
Mentor: David Young

Summary and goal:  The goal of the project was to add Teredo (RFC4380) support to the NetBSD kernel. Support for the client role was a top priority; relay and server roles, less so. Teredo helps hosts located behind a NAT router get IPv6 connectivity. The technology is complementary to 6to4, which does not work easily without a global IPv4 address.

Results:  The project did not succeed. The student did not install a "golden" Teredo implementation for him to compare with and test against his own implementation, so he did not have a testbed that was sufficient to test NetBSD in any Teredo roles. He was out of communication with his mentor for weeks at a time. He did write some code; another developer may be able to pick up where he left off, and finish the project.

Customizable install tool for NetBSD

Student: Zachary Wegner
Mentor: Tim Rightnour

Summary and goal:  The project was to split sysinst, the NetBSD install tool into a separate front and back end. This required: Defining a configuration file format for sysinst, and writing a parser for it. Splitting the front end off into a separate binary that can create the configuration file, and splitting the back end off into a separate binary that reads the file and installs the OS based on what it says.

Results:  The definition of a configuration file, and the binary which installs the OS based on that file are currently about 90% complete. Approximately 50-60% of the step to build the front-end configuration file builder are complete. Currently, the split sysinst is capable of reading a config file, and installing a system based on that file. As for the front end, it is currently capable of generating most of the config file that is required. There is still a bit of work to do to get the new sysinst to the point where it can replace the old one. Currently Zach is continuing to work on the project in the SourceForge repository while he awaits creation of his developer account that will grant him commit privileges. We intend to merge the code into NetBSD at some point, however, there are a number of technical and directory organization issues that we still have to figure out, due to the complexity of the installer, and the requirement that it work flawlessly.