Talk:Gossip protocol

message: Most of this page is extracted from a longer article I wrote for a special edition of Operating Systems Review (an ACM publication that doesn't impose copyright restrictions).  The longer article seemed a bit too detailed for a Wiki page, but it seemed to me that an encyclopedia treatment really should at least explain what a gossip protocol is, and isn't.
The references I included here are all cited in the longer article.  Some are papers I co-authored but most are by other folks.  I hope this isn't a violation of the POV/COI policies.  I'm not trying to sell anything and if people want to edit the article to reduce any perceived bias, go for it!
Are illustrations needed?  Ken Birman 12:59, 14 June 2007 (UTC)
thread_title:  Untitled 

message: For about six months now, this page has been tagged for Point of View, Original Research and Factual Accuracy.  After a brief period of discussion and editing activity, nothing has happened for six months.
Here's my question: as a matter of policy, do such tags remain on the page forever, or is there some procedure for agreeing to remove them?  It seems safe to assume that KPBotany, who placed them there, would not consider the matter resolved, since nothing substantive changed during this period.  However, perhaps that particular reviewer will never find comfort with this material -- such things do sometimes happen.  Ken Birman (talk) 18:04, 21 December 2007 (UTC)
replies: text: Protocol on wikipedia isn't always polite, professor. You'll notice on the history of this page, that some people concur with 
User_talk:KP_Botany in saying that this article needs work. I wouldn't know where to go with it. BrewJay (talk) 00:34, 14 July 2008 (UTC)
thread_title:  Protocol for removing tags? 

message: It's a multi-cast protocol. It hosts gossip. It's reliable. It's based upon internet structure that has existed for decades. Some companies are routing sound and files over it in real-time. Why reinvent the wheel? BrewJay (talk) 12:03, 4 July 2008 (UTC)
thread_title:  What's wrong with Internet Relay Chat? 

message: In the analogical lead of the article's beginning, that's what I get...that computers on the internet communicate directly along some wavelength. On some networks, like analog telephone (I'm not sure if that exists anymore), or packet radio, maybe they do, but there's no great element of randomness in it. A user picks a BBS, dials the number, BINGO, you hav a peer... Or, in packet radio, some ham scans the radio waves, recognizes digital, then tries to decode it using internet documents and utilities. Relaying QWK-based mail might happen a gossipy manner, but it was (is?) explicitly scheduled -- nothing like the random walk presented in this article. BrewJay (talk) 07:20, 9 July 2008 (UTC)
thread_title:  What's this about radio and direct communication? 

message: Here's an interesting story for you folks:
: … from an Amazon announcement Monday: (http://status.aws.amazon.com/s3us-20080720.html):
: 7:19 PM PDT We wanted to share a brief note about what we observed during yesterday's event and where we are at this stage. As a distributed system, the different components of Amazon S3 need to be aware of the state of each other. For example, this awareness makes it possible for the system to decide to which redundant physical storage server to route a request. In order to share this state information across the system, we use a gossip protocol. Yesterday, we experienced a problem related to gossiping our internal state information, leaving the system components unable to interact properly and causing customers' requests to Amazon S3 to fail. After exploring several alternatives, we determined that we had to temporarily take the service offline so that we could clear all gossipped state and restart gossip to rebuild the state.
: These are sophisticated systems and it generally takes a while to get to root cause in such a situation. We're working very hard to do this and will be providing more information here when we've fully investigated the incident. We also wanted to let you know that for this particular event, we'll be waiving our standard SLA process and applying the appropriate service credit to all affected customers for the July billing period. Customers will not need to send us an e-mail to request their credits, as these will be automatically applied. This transaction will be reflected in our customers' August billing statements. 
:: A subsequent news announcement explained that the problem wasn't the gossip protocol, per-se, but rather was caused by corrupted data that entered the system (somehow) and then spread through the gossip layer, contaminating everyone who saw it.  Here's the follow-on story (well, the relevant part of it)
:: We've now determined that message corruption was the cause of the server-to-server communication  problems. More specifically, we found that there were a handful of messages on Sunday morning that had a single bit corrupted such that the message was still intelligible, but the system state information was incorrect. We use MD5 checksums throughout the system, for example, to prevent, detect, and recover from corruption that can occur during receipt, storage, and retrieval of        customers' objects. However, we didn't have the same protection in place to detect whether this particular internal state information had been corrupted. As a result, when the corruption occurred we didn't detect it and it spread throughout the system causing the symptoms described above. We hadn't encountered server-to-server communication issues of this scale before and, as a result, it took some time during the event to diagnose and recover from it.  Ken Birman (talk) 12:29, 27 July 2008 (UTC)
thread_title:  Gossip protocol causes Amazon outage 

message: "Search strings known to A will now also be known to B, and vice versa. In the next "round" of gossip A and B will pick additional random peers, maybe C and D."
:Okay, so here we hav A communicates with B, and they exchange messages, carefully identifying their source with a message-ID, so that I don't need to count the number of unique messages. C and D do the same thing, at the same time as A and B. Later, A and C communicate, while B and D communicate the same way. In this way, you hav four nodes with the same information, and only two communication periods.
Period Nodes
1        2
2        4
3        8
...in thirty-two communication periods, you can hav over four billion nodes with information intended for broadcast. I wrote that method for QWK-packets, which were an early BBS standard for newsgroups. That standard didn't hav message-IDs, so my method had no potential (unless the advantage were worth seeking unique messages, for which I didn't hav an efficient method; now I would use a perfect hash. It still doesn't hav much potential, because hook-ups on the internet are largely manual and static. Notice that my method DEPENDS upon static addresses AND scheduled, regular communication periods. In practice, I don't see any hope for it, except perhaps within one box, where it has been blown away in practical applications, probably long before I even wrote it. 
:The problem is like one-bit adders strung together with shifters to make a multiplier. I think it illustrates what you're driving at professor, and I don't get logarithmic growth out of it, because the potential amount of information doubles at each period, so each scheduled period could be longer than the last. If, however, you limit the amount of information in an exchange period arbitrarily, and buffer it of course, then you can hav logarithmic growth, where the number of nodes with broadcast information exponentiates in relation to an arbitrarily low and fixed period.
I don't doubt that there are a lot of references for this in papers that have analyzed practicing it for use within a box, where it's easy to hard-wire timing, and when you stick randomness and unreliability into it, then you need some very special error correction protocol, indeed. I read that Intel spends a lot of money on noise control (crosstalk) within a CPU.BrewJay (talk) 16:42, 16 October 2008 (UTC)
thread_title:  This example reminds me of a scheme I wrote for telephony multi-cast... 

message: "The term convergently consistent is sometimes used to describe protocols that achieve exponentially rapid spread of information. For this purpose, a protocol must propagate any new information to all nodes that will be affected by the information within time logarithmic in the size of the system (the "mixing time" must be logarithmic in system size)."
:I see that growth functions are in here, and I'm having trouble parsing them. For example, "within time logarithmic in the size of the system". Should I rewrite that as "in time proportional to the logarithm of that system size"? I think I should do that and trim off the part in brackets, because I hav no idea what is being mixed in this context, and the mentioned function is the same.
:An interesting practical fault in this scheme you might like to note professor, is that exponential propagation rates are not possible on digital infrastructure. If, as you introduce in the leadup, there are (analog) frequencies to select and schedules to follow (as I demonstrate earlier, without rigor, schedules increase performance), then linear growth, proportional to the number of hosts, in the distribution time needed to for many-to-many news, for example, is feasible. I don't see how a logarithmic proportion can happen. Scheduled one-to-all is the simple way to do it in packet radio. There could be a protocol, for example, to get yourself a time-slot and frequency for digital broadcasts that would be heard. Same function: linear; as the community grows, so does the time needed for mirroring all hosts.
In the leadup, there is a mention of "sometimes". I will move to strike it out, because the word is hard to use in Physics. It is a troublesome thing to hav bits sometimes lost or to form an association for collaboration that is irregular. I understand that you want to demonstrate flexibility, reliability and performance. When a router goes down, then the metric for that router in neighbouring routers quickly registers unreachable. That propagates like gossip.
BrewJay (talk) 01:57, 21 October 2008 (UTC)
replies: text: Sure.  I actually meant this in a different sense, as in "some people use the term, but others don't".  But you can strike it.  BTW, everyone calls me "Ken" even in my classes.  I know that in much of the world it seems odd to call a professor by anything but the title, but I'm fine being just Ken, so feel free to be a bit less formal!  Ken Birman (talk) 12:22, 22 October 2008 (UTC)
thread_title:  Exponential propagation rates and logarithmic growth. 

message: I came here searching for "worst protocol ever" (half-expected a redir to SOAP). The search function seems to have understood "worst article ever". Maybe I mistyped.  — Preceding unsigned comment added by 194.228.11.192 (talk) 06:03, 2 September 2016 (UTC)
thread_title:  Something wrong with the search function 

message: The lead right now is a metaphor sandwich, with no technical depth. Where's the beef?  — MaxEnt 18:41, 7 January 2018 (UTC)
thread_title:  Metaphor sandwich 

message: Almost an unlimited number of "Gossip" variants could be mentioned. They are not WP material unless supported by current use and/or citations.
In that vein, I have also removed "Spacial Gossip". Although explained, there were no examples given of current use and also no valid references. See 'Unsupported Claims' section for a similar issue.  — Preceding unsigned comment added by 50.68.57.160 (talk) 16:50, 26 November 2018 (UTC)
thread_title:  Of the Many Types of Gossip Protocols