PgCon 2017 Developer Meeting

A meeting of the interested PostgreSQL developers is being planned for Tuesday 23 May, 2017 at the University of Ottawa, prior to pgCon 2017. In order to keep the numbers manageable, this meeting is by invitation only. Unfortunately it is quite possible that we've overlooked important individuals during the planning of the event - if you feel you fall into this category and would like to attend, please contact Dave Page (dpage@pgadmin.org).
Please note that the attendee numbers have been kept low in order to keep the meeting more productive. Invitations have been sent only to developers that have been highly active on the database server over the 10.0/9.6 release cycles. We have not invited any contributors based on their contributions to related projects, or seniority in regional user groups or sponsoring companies.
As at last years event, an Unconference will be held on Wednesday for in-depth discussion of technical topics.
This is a PostgreSQL Community event.
== Meeting Goals ==
* Define the schedule for the 11.0 release cycle
* Address any proposed timing, policy, or procedure issues
* Address any proposed Wicked problems
== Time & Location ==
The meeting will be:
* 9:00AM to 12PM
* DMS 3105
* University of Ottawa.
Coffee, tea and snacks will be served starting at 8:45am. Lunch will be after the meeting.
== RSVPs ==
The following people have RSVPed to the meeting (in alphabetical order, by surname):
* Aleksander Alekseev
* Oleg Bartunov
* Joe Conway
* Jeff Davis
* Andrew Dunstan
* Peter Eisentraut
* Andres Freund
* Stephen Frost
* Etsuro Fujita
* Peter Geoghegan
* Kevin Grittner
* Robert Haas
* Magnus Hagander
* Álvaro Herrera
* Kyotaro Horiguchi
* Amit Kapila
* Haribabu Kommi
* Alexander Korotkov
* Tom Lane
* Amit Langote
* Heikki Linnakangas
* Noah Misch
* Bruce Momjian
* Thomas Munro
* Dave Page
* Michael Paquier
* Masahiko Sawada
* Teodor Sigaev
Apologies
* Simon Riggs
== Agenda Items ==
* Upgrading PostgreSQL without a downtime. (Aleksander Alekseev)
* Commit fest management (Michael)
* Intellectual Property Issues (Andres)
* Recognizing Contributors (Stephen)
* Release notes scope, and giving credit (Peter E.)
* Web site design (Peter E.)
* 64-bit xids (Alexander Korotkov)
* Adopting an indent fork (Álvaro Herrera)
* Please add suggestions for agenda items here. (with your name)
==Agenda==
Time 	Item 	Presenter
09:00 - 09:10	Welcome and introductions 	Dave Page
09:10 - 09:20	PG 11 release and commitfest schedule 	Dave Page
09:20 - 09:40	Commit fest management 	Michael
09:40 - 10:00	Upgrading PostgreSQL without a downtime	Aleksander Alekseev
10:00 - 10:05	Intellectual property issues 	Andres
10:15 - 10:30	Web site design 	Peter Eisentraut
10:30 - 10:45	Coffee break 	All
10:45 - 11:00	Release notes scope, and giving credit 	Peter Eisentraut
11:00 - 11:20	Recognising contributors 	Stephen
11:20 - 11:40	64-bit xids 	Alexander Korotkov
11:40 - 11:50	Adopting an indent fork 	Álvaro Herrera
11:50 - 12:00	Any other business 	Dave Page
12:00 	Lunch 	
== Minutes ==
=== Welcome and introductions ===
Attendees:
* Aleksander Alekseev
* Joe Conway
* Jeff Davis
* Andrew Dunstan
* Peter Eisentraut
* Andres Freund
* Stephen Frost
* Etsuro Fujita
* Peter Geoghegan
* Kevin Grittner
* Robert Haas
* Magnus Hagander
* Álvaro Herrera
* Kyotaro Horiguchi
* Amit Kapila
* Haribabu Kommi
* Alexander Korotkov
* Tom Lane
* Amit Langote
* Heikki Linnakangas
* Noah Misch
* Bruce Momjian
* Thomas Munro
* Dave Page
* Michael Paquier
* Masahiko Sawada
* Teodor Sigaev
=== PG 11 release and commitfest schedule ===
Haas proposed using the same schedule as last year, only shifting the
dates forward by one year. Measure passed unanimously.
Timing of the release branch is not yet clear.
=== Commit fest management ===
(Paquier) I seek feedback on recent CF management. At the end of each CF, I
classify each patch's status and send a short email to the patch's thread. Is
that spam? (Haas) No, it's good; some authors read their threads only. Don't
decide the fate of a patch on an additional thread that doesn't include the
author. CF manager does need to push other CF participants.
(Freund) Don't always move patches to the next CF; use Returned with Feedback
instead. Patches have traversed 4+ CFs without changing. (Haas) If something
is or should be Waiting on Author at CF end, use Returned with Feedback. When
authors have submitted an updated patch at the end of a CF, Paquier has moved it
to the next CF. That is a good move. (Lane) I worry about patch authors not
familiar with the CF process, who may not know to resubmit after Returned with
Feedback. I propose moving the patch at the end of one CF, then closing it if
the next CF starts with no state change.
(Freund) Returned with Feedback breaks up the history, because the next
submission will create a new entry. (Eisentraut) That is a tooling problem.
(Paquier) CFs used to have 100 patches max, but they now see 250. Process
struggles to keep up. (Haas) If we ever kept up with patch flow, we don't now.
(Grittner) Patch repeatedly moving from one CF to the next can mean nobody was
interested, or nobody competent had the time. (Freund) We're bad at giving
early feedback that a change is not wanted. (Lane) For every patch, at least
one person (the author) found it interesting. (Geoghegan, Lane) We don't
outright reject very much. (Kapila) Each case is controversial. (Misch) Most
almost-rejections are due to implementation, which we can't foresee early.
(Eisentraut) When we move a patch to the next CF, the reviewer may not be aware.
Should we clear the reviewer slot on move? (Haas) Reviewer list in CF app is
often very wrong. (Paquier) Eliminate CF app tracking of reviewer? (Kapila)
Reviewer field is helpful as a target for CF manager nagging. (Haas) Don't
remove it, but don't trust it. (Lane) Should the CF manager aggressively remove
an inactive reviewer? (Haas) I support clearing the field at CF end.
(Grittner) When removing a reviewer, notify that person by email. (Linnakangas)
The open items process, with Misch emails, has worked perfectly. (Eisentraut) I
don't care about nag emails. Patches get stalled ~6 weeks because the listed
reviewer is idle; other reviewers focus on entries with blank reviewer.
(Paquier) The wiki's CF checklist is very outdated. I propose to rewrite it.
Will add useful CF app links to that page or to the patch submission page.
After doing those updates, I will request feedback from -hackers.
=== Upgrading PostgreSQL without a downtime ===
(Haas) As a technical topic, this is out of scope for this meeting. Save it for
the unconference. (Page) Okay to discuss technical decisions such as openness
to on-disk format changes. Technical discussion did work well in Brussels.
(Korotkov, Alekseev) The community is conservative. For example, it uses Perl.
We plan to allow v10->v11 upgrades but not direct v10->v12 upgrades. Is that
okay? (Misch) I don't see a problem with this one-version upgrade window.
(Hagander) Agreed; it's okay to need five upgrades to cross five versions.
(Haas) Biggest thing that won't work is system catalog changes. (Geoghegan)
Logical replication solves many of the problems.
(Page) With SQL Server, you can just upgrade the binaries and restart.
(Alekseev) I'm defining "zero downtime" as "indistinguishable from failover to a
standby". (Grittner) Not proposing to preserve TCP connections? (Alekseev)
Correct, not proposing that. (Geoghegan) Suggest starting with minor release
updates. (Haas, Grittner) Under this definition of zero downtime, that's
already possible.
(Haas) I posted a pie-in-the-sky design for major releases. Lane found flaws.
(Linnakangas) This will differ from the multiple-page-versions approaches, which
still assumed use of pg_upgrade.
(Page) Is it acceptable to require replication and a second server?
(Herrera) Will every catalog change require code to support this? (Alekseev)
No. For example, we just won't drop old catalog columns. (Davis) Challenging
thanks to our use of C structures and offsets to access catalogs.
(Haas) It will never work to perform physical replications across major
versions. Could have v11 binaries recognize and convert a v10 cluster, but
cannot have a v10 replica with a v11 master. (Freund, Momjian) Yes; the
protocol is compatible, but the payload is not. (Linnakangas) It's possible but
far more complicated and hard to test.
(Misch) Propose conclusion that the group has no objections to this proceeding.
We've provided various challenges to cover in the design.
(Lane) Possible alternative is to spend time making pg_upgrade faster. (Freund)
ANALYZE statistics are the hard part. (Haas, Eisentraut) pg_upgrade fails for
hard-to-predict reasons.
=== Intellectual property issues ===
The group voted to exclude this topic from the minutes.
=== Web site design ===
(Eisentraut) Site is stuck in the past. We have occasional pushes to hire
someone or otherwise fix it. (Frost) We did a bunch of work that didn't get
merged. Showed it off at pgconf.eu 1.5 years ago. Sarah Conway of Crunchy Data
is leading the design effort. Someone needs to reimplement that on a different
Django version. (Momjian, Haas) This lacks project management. Who, if anyone,
will do the remaining work? (Frost) I am targeting completion this fall.
(Eisentraut) If I email pgsql-www every two weeks, is that annoying enough?
(Misch) What's actually wrong with the current site? (Eisentraut) Looks bad on
small devices. (Geoghegan) It creates a bad impression for new users.
(Page) Incremental changes can be hard; changing the font size could lead to
months of bikeshedding. (Hagander) Partial migration is undesirable, because it
would leave the site with a blend of different designs.
(Hagander) Updating the wiki is more urgent. (Eisentraut) Is there a backlog of
infrastructure issues? (Hagander) Just the wiki and the mailing lists.
=== Release notes scope, and giving credit ===
(Eisentraut) Some items attributed to me warrant credit to important reviewers.
(Linnakangas, Eisentraut) How should we treat performance improvements in the
release notes? (Momjian) I can easily recognize significant feature additions,
but it's harder to tell the significance of a performance improvement. I want
to include changes that are actionable, that allow new uses of PostgreSQL. From
1200 commits I wrote 200 release note entries, some of those associated with
more than one commit. Adding commit hashes as SGML comments has been a big win.
I expect changes to initial notes; for v10, the editing thread was shorter than
usual. (Freund) I was dissatisfied with the original treatment of performance
patches and the general statement about their value. "Will it run faster?" is
the #1 thing for my users. However, I'm content with the release notes as they
stand today. (Haas) Committers have an action item to write commit messages
that explain the change's significance. (Momjian) Commit messages were really
good. (Haas) Hard to write notes for Lane improvements that address rare query
shapes. (Linnakangas) Should consciously include an end-user-visible example.
(Grittner) Most releases improve two kinds of performance. First, they improve
concurrency: they move the performance "knee" to the right (more clients) and
make its decline shallower. Second, they make certain workloads suffer less
bloat.
(Conway, Haas) Including mailing list links in commit messages has been useful.
(Haas) Do we keep including author names in release notes? (Freund) Linking to
the commit would suffice for credit. (Eisentraut) Should we move all names to
the end? (Momjian) Names at the end is okay if we do link to commits. (Haas)
Linking to commits would be useful. (Momjian) Use javascript to show/hide all
commit references, and/or show them on mouseover. (Dunstan) PDF docs won't
handle that as well. (Eisentraut) Implementing dynamic behavior in every doc
format is a lot of work.
(Momjian) v10 has 10-20% fewer release note items than recent releases, but it
has big items. (Andres) That will change next release.
(Momjian) For 9.2 and 9.3, Josh Berkus and others gave feedback that listing
reviewers was too much. (Eisentraut) Nonetheless, we should list them in the
release notes. (Misch) I realize it's not the community consensus, but I remain
supportive of listing reviewers at the bottom of the release notes. (Dunstan,
Haas) We wouldn't mind that. (Dunstan, Misch) We don't credit one-line
reviewers in our commits, so they'd be appropriately excluded from release
notes. (Linnakangas) We could even credit everyone who send a mailing list
post. (Geoghegan) I favor inclusion: not one-line reviewers, but almost
anything else. People don't read notes end-to-end, anyway. (Eisentraut)
Propose we
collect all names mentioned in commit messages and stick them at the end of the v10 release notes.
I will do the work. Measure passed unanimously.
=== Recognising contributors ===
(Frost) We have competing proposals for updating the web site contributor list.
Need to define criteria. Could even just use the list Eisentraut makes for the
release notes. (Haas) Don't want to add that many. When I proposed a list, I
had forgotten that last year's meeting had commissioned another effort.
Committed CF entries dominated the Frost proposal, and that's a problem.
(Frost) I looked at email size as a way to recognize non-code contributions.
(Haas) We do want to recognize non-code contributions.
(Freund) We're letting perfect be the enemy of good. (Page) We should vote
yea/nay on a list presented so far, and move on. (Haas) Analyzing 2016 commits
took ~8hr. Fine to automate if we can, but automation is inessential. I'm
willing to spend 24hr analyzing satellite projects. Frost has blocked each of
my proposals. (Page) Your time is better spent on code. (Haas) I thought that
for five years, too. (Page) Propose taking Conway's list, discussing anything
needed, and adopting it.
(Page, Haas, Geoghegan) Numbers should be a starting point; they should not
dictate the final answer. (Haas) Best to remove CF data from the pile.
Poll: ~40% of this room is not on the contributors list.
(Geoghegan) We'd be unanimous on recognizing 10-20 people.
(Linnakangas) Who will physically update the site? (Conway, Lane) Suggest
converting Haas list to a web site diff and sending it to
pgsql-private-committers@ for approval. (Haas) Expect more cycles of update.
For example, I only classified people as past contributors when 100% confident.
Complaints are inevitable.
(Haas) Propose having Frost and myself sit down after this meeting and agree on
something. (Momjian) I want the site updated by Thursday. (group conclusion)
Haas and Frost may proceed to update the list, letting let this happen
quickly.
=== 64-bit xids ===
(Korotkov) Postgres Pro has a patch for this. Some customers have problems not
fixed by 9.6 freeze work. No access to production systems for seeing details.
(Misch) Community would need to see the good-case and worst-case benchmarks.
(Grittner) This won't fix vacuuming to reclaim space.
(Lane) Eliminating freezes is attractive.
(Geoghegan) What happened to the Linnakangas version of this? (Linnakangas)
It's unfinished.
(Haas) We know there's more to improve here, but we don't know what exactly
remains.
(Haas) I will vote against widening xmin/xmax on disk. The Linnakangas approach
didn't have that problem. (Linnakangas) Don't want on-disk format changes.
(Linnakangas) Do problems remain despite free space map and freeze map?
(Freund, Geoghegan) Yes. (Freund) I see anti-wraparound vacuum problems in 9.6
three times per week, in response to configuration changes. Spending 1.8B XIDs
in two days is not the worst I've seen. (Geoghegan, Freund) Nobody cares about
clog size. (Haas) People do care, but not the same people who care about
anti-wraparound vacuum.
=== Adopting an indent fork ===
(Lane) Considering adoption of Piotr Stefaniak indent fork. I studied the diff
it causes and liked nearly everything. Should we embed it in the PostgreSQL
tree, or should we give it a standalone repository? In-tree lets us correlate
changes, and out-of-tree lets us use the latest version on all branches. (Haas)
Weird to cram a standalone tool into the main tree. (Misch) Similar to TAP
support modules. (Lane) Similar to much of src/tools.
(Freund) Great if every contributor can easily run indent. (Eisentraut)
Developers will figure it out easily, relative to other challenges of starting
development like installing dependencies and configuring one's editor. (Freund)
Mainstream OSs package all dependencies, so that part is easy.
(Haas) We should reindent every week or every day. By end of cycle, the tree
has accrued lots of divergence. It's hard to commit indent-clean code to a file
that is already not indent-clean. "make indent" would be great.
(Eisentraut) If we switch indent implementations, I want one with upstream
maintenance. (Freund) Author of proposed replacement is a FreeBSD committer and
plans to submit indent code to FreeBSD. (Misch) We won't pull from upstream
automatically; syncs will be more like timezone code syncs. (Freund) The
replacement is already more-maintained than pgindent. (Lane) I vote not to wait
for better maintenance. (Haas) If Lane will cover any problems, let it happen.
Proposed tool does improve things.
(Eisentraut) Happy leaving v10 as-is (pgindent).
(Lane) Distinct issues (1) whether to switch indent tools and (2) how to
distribute indent tool code. (Misch) Also (3) how often to reindent.
=== Any other business ===
(Misch) Congested agenda today. Should we meet longer next year? (Haas)
Schedule was based on an unconference starting after the meeting, which we no
longer do. But we wouldn't survive a 09:00-17:00 meeting. (Hagander) Perhaps
09:00-14:00?
(Page) How should the other developer meetings compare to this one? Brussels
has been heavy on technical topics. (Haas) Unconference is better for technical
topics. We used to have lots of narrow-interest technical topics that boiled
down to the presenter speaking at a non-participating group.
(Linnakangas) When linking to mailing list posts in commit messages, I use a
postgresql.org URL, but most use postgr.es. (Eisentraut) Should we link to the
whole thread or to a single message?
Category:PGCon
Category:Developer Meeting