OpenZFS Developer Summit 2013 planning

This page contains information used to plan the event. For the proceedings of the event, including video, slides, and event report, see OpenZFS Developer Summit 2013

Tuesday, November 19th
Note that the optional, off-site dinners on Sunday and Tuesday are at your own expense.

Location
The Delphix office in San Francisco:

604 Mission Street, 4th floor, San Francisco, CA 94105

Use the keypad to call Delphix and you will be buzzed in. Google maps has pretty good directions. The easiest way to reach the Delphix SF office is by public transportation. Transportation options, easiest listed first:
 * BART to the Montgomery St. Station + 3 minute walk
 * Caltrain to the 4th & King station + 25 minutes walk
 * Caltrain to the 4th & King station + bus (routes 10, 30, or 45)
 * drive + park in a lot (costs about $25/day e.g. 41 Tehama Street, 509 Howard Street, or 55 Hawthorne Street)

Discussion topics
Note: If you are in the room, you are potentially a speaker. This will be done in unconference style, so every attendee should come prepared to present or otherwise lead a discussion or session of up to 45 minutes. The most important things to discuss in this setting are:


 * 1) Contentious topics that we need to get widespread agreement on
 * 2) Reports of work in progress that need input from others

Proposed topics

 * OpenZFS code repository (Matt)
 * what code changes are needed to make this a success
 * process & procedure (code review, testing, committers)
 * Performance on full or fragmented pools (George, Ned, Alfred)
 * block allocator improvements
 * Portable implementation of FMA functionality for Linux & FreeBSD (e.g. for hot spare activation) (Brian)
 * OpenZFS community organization (Karyn, Jordan, Matt)
 * Storage tiering (Boris)
 * How to investigate ZFS performance problems (Adam)
 * Test suite (John K)
 * ZFS Channel Programs (Chris, Max G, Matt)
 * how to examine the ZFS on-disk format with MDB (Max B)
 * Interactions with virtual memory subsystem (Brian, Jorgen)
 * dealing with swap on zfs
 * how the VM system handles large memory allocations
 * Scalability on large systems, known bottlenecks (Kirill)
 * How to educate new ZFS developers (Max G)
 * Disk management - identification, drive spin down/up (John L)
 * Resumable send/recv (Boris, Max G, Chris)

Topics that were discussed

 * Platform Panel (w/ representatives from illumos, FreeBSD, Linux, and OSX)
 * Platform-independent code repository (Matt Ahrens)
 * Storage tiering (Boris Protopopov)
 * Vendor Panel (all represented companies sharing their work)
 * Community planning (Karyn Ritter)
 * Channel programs (Chris Siden & Max Grossman)
 * Test coverage (John Kennedy)
 * Performance investigations (Adam Leventhal)
 * Performance on full/fragmented pools (George Wilson)
 * Scalability (Kirill Davydychev)
 * Virtual memory interactions (Brian Behlendorf)
 * Multi tenancy (Rob Mustacchi)
 * Examining on-disk format (Max Bruning)

Hackathon
The goals of the hackathon are: Therefore, we should plan to work in small teams of 2-3 engineers. The team members could be pair-programming, splitting up the work into discrete tasks for each person, or advising / implementing.
 * to get people working with ZFS engineers from outside their normal circles
 * to start work on some of the projects we'd like to accomplish

Hackathon projects
These projects were worked on at the hackathon:


 * Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)
 * channel programs: pass instruction stream into kernel
 * ZFS send progress reporting with "set proc title"
 * dedicated kmem_cache for dedup table entries
 * larger (1MB) blocksize
 * compressed ARC
 * upstreaming dbuf_user & dbuf state machine (spectra DMU changes)
 * limit DDT size
 * onexit for osx

Hackathon ideas
''Add your idea and your name below. Be prepared to explain your idea at the beginning of the hackathon and enlist/entice others to help with it.''
 * Userland ioctls: support for send/recv (fd operations over the wire) User:mahrens
 * larger (1MB) blocksize: shouldn't be too hard if we ignore performance impact User:mahrens
 * dedicated kmem_cache for dedup table entries
 * stdin / stdout buffer to improve performance of zfs send and receive
 * ZFS mechanism to override devices' reported sector size (512 vs 4k) (alex @ delphix)
 * feature flags enhancements for send streams User:csiden
 * set owner of all files within a filesystem with a ZFS property User:csiden
 * dynamic taskqs for ZIO User:ahl
 * towards Oracle Solaris ZFS compatibility User:ahl
 * RAIDzN -- full, arbitrary RAID-7 User:ahl
 * Testrunner suite running on other platforms (John Kennedy)
 * or at least identify what is preventing it from running

Registration
We are expecting everyone will be there to contribute/participate.

Sponsors
The event was fully sponsored! Thank you to our sponsors:


 * Breakfast (2 days) - Sponsored! Thank you Cloudscaling and iXsystems
 * Lunch  (2 days) - Sponsored!  Thank you Sage Cloud, Spectra Logic and FreeBSD Foundation
 * Coffee Breaks (2 days) - Sponsored! Thank you iXsystems
 * Beer Bash (day 1) - Sponsored! Thank you DDRdrive
 * T-Shirts - Sponsored!  Thank you Delphix
 * Video Recordings - Thank you Joyent
 * Cash Donation for expenses - Thank you Nexenta and High-Availability

What do you get for sponsoring:


 * Your name/link on this page!
 * Thank you mention during the opening and closing
 * Space to put your pop-up banner at the conference venue
 * An opportunity to be known as a ZFS supporter in a very talented and influential community.