NetBSD 10: 3 Years in the Making

Nia Alarie, EuroBSDCon 2022

Notes for this talk

  \-__,------,___.
   \        __,---`  NetBSD/evbarm efiboot (arm64)
    \       `---,_.  Revision 2.10 (Thu Aug 19 12:13:37 UTC 2021)
     \-,_____,.---`  Memory: 512 MB
      \
       \
        \

Release flow (not to scale!)

HEAD------------------------------------------------------------[...]
              |                                         |
              |                                         |
              |                                         |
              |                                      netbsd-10 ---|
              |                                                  10.0
              |
              netbsd-9 ------|----------|---------|-------------[...]
                            9.0        9.1       9.2

[...]--|--------------------------|--------------------fin
        8.1                        8.2

Quality assurance tools

                         build.sh
                            |
                            |
      ----------------------|
     /                      |
Static analysis             |
(-Werror, lint, Coverity...)|
                            |
                          Anita
             (9+ archs - QEMU/NVMM, gexmul...)
                            |
                       ------------------  
                      /                  \
                 Rump kernels      Sanitizers
                      |            (KUBSAN, KASAN, KMSAN...)
                      |            and KASSERT()
                     ATF                 |
                10,000+ tests            |
                                   Kernel fuzzers (syzbot)

Late 2019

Andrew Doran begins reworking critical subsystems of the NetBSD kernel.

scheduler: Improve performance and interactivity. [ad 20191201]
vfs: Reduce false sharing and lock overhead during normal operation.
	[ad 20191201]
uvm: Replace global counters with per-CPU counters, significantly
	reducing system time on workloads that are VM-system heavy.
	[ad 20191216]
scheduler: Make the scheduler topology aware.  It understands and
	makes good use of HyperThreading/SMT, spreads the load evenly
	across different physical CPU packages, and can effectively
	run asymmetric systems with slow and fast CPUs [ad 20200113]

By November 2020, performance on multi-core systems almost doubles.


Trouble a'brewing

current-users@, early 2020:

Many bugs found with syzbot, then fixed.


Lessons learned?

For finding and fixing bugs early?


January 2020 - Great Driver Cleanup


Driver Cleanup Results

Gone: all FDDI, all HIPPI, all Token Ring, SMBFS, strip(4), de(4), uyurex(4), uyap(4)

Kept: sl(4) (serial line IP), se(4) (SCSI Ethernet adapter), pf(4) (the deprecated packet filter)

It went well compared to previous removal proposals!


Summer 2020

"We're all excited about the fabulous progress that -current has made, especially in performance but it's also quite obvious that it has serious issues and we are far from being able to branch netbsd-10"


Summer 2020: random saga begins

Unforseen problems: "Entropy error blocks lang/python38 installation"


2020 in Summary


May 2021: CFARGS audit and kernel APIs

NetBSD 9.x:

Verdict:


August 2021: CFARGS solution

Old example:

    c->c_dev = config_found(sc->sc_dev, &pa, pciprint,
        CFARG_SUBMATCH, config_stdsubmatch,
        CFARG_LOCATORS, locs,
        CFARG_DEVHANDLE, devhandle,
        CFARG_EOL);

New example:

    c->c_dev = config_found(sc->sc_dev, &pa, pciprint, 
        CFARGS(.submatch = config_stdsubmatch,
               .locators = locs, 
               .devhandle = devhandle));

DRM update


DRM update challenges

Once again, KASSERT critical to get useful information from users for debugging...

How can we help users test large kernel changes?


2021 in Summary


Early 2022

Lessons learned from pkgsrc:


Mid 2022

Foot placed down by releng@.


Unfinished business


Conclusions

NetBSD 10.0:

WORTH IT. It's been an amazing journey! Thanks everyone!