OpenZFS Developer Summit 2014

Photos
Photos of the event can be viewed on the OpenZFS Google+ page.

Presentations
<!--

Location
Note: Day 1 and Day 2 of the event will take place in different locations a 10 minute walk apart.

On Monday, November 10, 2014 the event will take place at the Children’s Creativity Museum

221 Fourth Street, San Francisco, CA 94103. Behind the Carousel at the corner of Fourth and Howard.

On Tuesday, November 11, 2014 the event will consist of the hackathon and related activities and will take place at the Delphix San Francisco Office

604 Mission Street, 4th floor, San Francisco, CA 94105

Projects started at the hackathon

 * Channel Programs - continuation of work from last year's hackathon
 * See project page, slides and 2013 and 2014 videos describing project
 * Chris and Paul - (Best in show winner) Demo of several working channel programs, lots of testing yet to do.
 * Upstream zfs-make - tool to quickly build zfs on illumos
 * Justin and Andiry - 5 new issues upstreamed for review, including a performance enhancement aggregating sync and async operations
 * Dan McDonald brought zmake to illumos
 * Steve, Chip, Brian B. - Building zfs on illumos
 * Poor mans compressed send (decomp when recv)
 * Matt Ahrens - Working prototype, some work remains (e.g. send flags)
 * Sha 512/256
 * Saso and John Mark - Implemented Blake2 hash instead. Saso also revived a SCSI unmap patch
 * Rip out scrub throttle, queue mb not ios
 * George Wilson - Mostly finished, needs testing
 * One man page per subcommand
 * Manoj - Completed this work
 * Use GRUB2 to boot Linux from a root pool with multiple vdevs (
 * Richard Yao - Done (as of plane flight), submission to upstream will follow shortly
 * https://bpaste.net/show/d32e6ff0661f
 * vdev kstats (Richard)
 * Richard and Dan - Added several new per dataset zil kstats
 * object count per user in DMU
 * Isaac and Manoj - Fully vetted design, and implementation begun.
 * improve arc_kmem_reap
 * Tony and George - Made progress on this, as well as improving the amount of memory consumed by arc buf headers when using l2arc.
 * quickly creating fragmented pools
 * Tony and George - Use a passed in histogram to create desired fragmentation level - near code complete
 * reconstruct space_maps
 * Albert - Has this nearly working
 * large dnodes
 * Ned worked on this, solving several crashes - still testing for edge cases
 * open-zfs infrastructure in docker (flocker)
 * Luke - made progress on the mailman and website infrastructure
 * fix OS-2313
 * Bryan Dan and Robert worked on a zil_close/spa_sync race that causes intermittent panics and hangs
 * fix zpool commands hanging when we have suspended pools
 * Don and Brian B. - Created a fix for this bug
 * Data collection using statistical methods for weighting recorded data by arbitrary time range
 * Alex and Prakash - A working demo measuring spa activity
 * Add ziltest to zfstest
 * John and Don - got this working, also added a zpool subcommand to list unused disks.

Ideas for next hackathon

 * preferred allocation alignment (george)
 * JSON command output
 * separate manpages for each zfs subcommand
 * update build instructions
 * iokstats for sync vs async
 * iokstats for each dataset
 * fix race condition in zpool import device detection
 * zfs set fsid_guid= (and/or include fsid_guid in send stream)
 * new CLI option to enable send features: zfs send -x embedded,largetblock or -x all
 * JSON command output
 * File cloning with implicit indirection
 * print usage message for specific property
 * zfs help command
 * compile ZFS on Linux with -std=gnu99; does it work?
 * storage pool aging
 * specify histogram of free segment sizes
 * allocator chooses locations to satisfy histogram
 * "get next file" syscall
 * iterates in object # order, opening next file; zfs prefetches dnodes
 * could speed up operations that need to operate on every file in a filesystem, e.g. "chmod -r"
 * Porting Party - learn how to contribute changes to a new platform
 * performance analytics framework - record performance, store & display (dtrace + graphite?)
 * USE method data collection framework for performance analysis (http://www.brendangregg.com/USEmethod/use-solaris.html)
 * Illumos grub2
 * Someone was already working on this before the hackathon.
 * https://github.com/tsoome/illumos-gate/wiki/Grub2---part-1,-the-preparation
 * Clean up embedded zil code
 * UTF-8 dataset & poolname (Richard)
 * zfs_core collaboration (Richard)
 * collectd agents (Richard)
 * scrub/resilver progress (Richard)
 * promote prefetch reads to sync reads
 * make zfs recv less trusting
 * raidz1 for single device
 * static analysis tools for zfs
 * dynamic analysis for ztest
 * improved documentation for developing zfs on linux
 * runtime lock analyzer
 * make dtrace probes in linux to work with lttng
 * per zone I/O QoS
 * convert AVL trees to B-trees or hash tables
 * cross platform xattr
 * use refcount_t for v_count

Harassment Policy
OpenZFS Developer Summit is dedicated to providing a harassment-free conference experience for everyone, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age or religion. We do not tolerate harassment of conference participants in any form. Sexual language and imagery is not appropriate for any conference venue, including talks. Conference participants violating these rules may be sanctioned or expelled from the conference at the discretion of the conference organizers.

For information about last year's event, see the OpenZFS Developer Summit 2013 page.