[SPARK-2677] BasicBlockFetchIterator#next can wait forever

------------------ 原始邮件 ------------------
  发件人: "Josh Rosen";<notifications@github.com>;
 发送时间: 2014年8月12日(星期二) 中午12:24
 收件人: "apache/spark"<spark@noreply.github.com>; 
 
 主题: Re: [spark] [SPARK-2677] BasicBlockFetchIterator#next can waitforever (#1632)

 

 
In core/src/main/scala/org/apache/spark/network/ConnectionManager.scala:
>                } >                case None => { > -                throw new Exception("Could not find reference for received ack message " + > -                  message.id) > +                /** > +                 * If isAckTimeout == true, Future returned from sendMessageReliably should fail > +                 * and finally, FetchFailedException is thrown so in this case, we don't need > +                 * to throw Exception here > +                 */ > +                if (!isAckTimeout) {  
Since we delete the MessageStatuses of messages that fail with timeouts, I guess we have no way to distinguish between an ACK for a message that we didn't send and an ACK for a timed-out message. I guess that isAckTimeout is used to strike a reasonable compromise in which we'll detect errors if no timeouts occur.
 
It might be a good idea to logWarning if we receive an ACK after we've timed out; this might help when debugging: if the ack timeout is set too low, warning messages would appear in the logs.
 
—
Reply to this email directly or view it on GitHub.