User1: User2
User1: lp://staging/~mwhudson/linux/trunk  	Started: 48 minutes ago  	Last heartbeat: 14 seconds ago
User1: 2010-04-15 22:54:11 INFO    49969129 bytes transferred | Fetching revisions:Inserting stream
User1: 2010-04-15 22:55:11 INFO    43484334 bytes transferred | Fetching revisions:Inserting stream
User1: 2010-04-15 22:56:13 INFO    35991244 bytes transferred | Fetching revisions:Inserting stream
User1: 2010-04-15 22:57:14 INFO    31731071 bytes transferred | Fetching revisions:Inserting stream
User2: User1 i meant like "log in and run bzr st", sorry
User1: User2 hah, ok
User1: User2
User1: importd@strawberry:/srv/importd.staging.launchpad.net/staging/launchpad$ bzr st
User1: unknown:
User1: lib/canonical/launchpad/apidoc/wadl-staging-1.0.xml
User1: lib/canonical/launchpad/apidoc/wadl-staging-beta.xml
User1: lib/canonical/launchpad/apidoc/wadl-staging-devel.xml
User2: User1 oh ok
User1: revno 9233, if that is at all interesting
User2: User1 can you run bzr merge  http://bazaar.launchpad.net/~mwhudson/launchpad/bzr-git-improvements ?
User2: if it has conflicts, please revert it
User1: User2 Warning: criss-cross merge encountered.
User2: User1 hmm
User1: still going though
User1: ah, 3 conflicts
User1: i shall revert
User2: thanks
User1: welcoem
User2: i guess i'll just wait until all the changes make their way through to db-stable
User2: 10721 Launchpad Patch Queue Manager	2010-04-15 [merge]
User2: [testfix][rs=me][ui=None] Fix some tests broken in the branch that
User2: changes the default PPA quota
User2: but not all of them!
User2: guys!
User3: User2 Ew. What's left?
User2: User3  lib/lp/soyuz/browser/tests/archive-views.txt
User2: i have a branch in pqm fixing it
User3: Ah.
 * mwhudson lunches early
User4: User5 can you please tag any merge queue bugs with merge-queues?
User5: User4 sure
User4: ta
User5: User2 you said my 'cannot set status to 'code-failed-to-merge' bug was trivial'
User5: User2 care to point me at it ?
User2: User5 i think that was thumper
User4: ?
User4: BranchMergeProposal.set_status
User5: User2 well, the plural 'you'
User5: User4 yeah
User4: or setStatus
User5: bug 561160
User6: Bug #561160: API: 'Code failed to merge' setting doesn't work <code-review> <merge-queues> <trivial> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/561160>
User5: I pinged mwhudson because thumper was going out, I thought ;P
User5: I'd just like a pointer at what to hit on, given its marked trvial
User4: trivial for me
User4: perhaps not trivial for you :)
User5: User4 so, gimme a couple pointers, and we'll see
User4: User5 setStatus needs a conditional to set mergeFailed
User5: where do the tests live ?
User4: lp/code/model/tests
User4: test_branchmergeproposal.py
User5: User4 Looks to me like setStatus has a bug, if you do setStatus(Rejected) when it was Queued, it won't dequeue
User4: User5 it may well do, we haven't extensively tested queues
User2: User7 hey, could you trigger an update of tellurium and the staging code import slaves pls?
User7: User2 sure, gimme a bit...
User5: User7 so, we haz keyfile ? :)
User5: User7 I'm keep to get to the point that we find new rt tickets to file
User5: User7 at which point stopping won't block things
User2: User7 update in progress? if you don't have time feel free to tell me to go away :-)
User7: not yet....
User5: User4 you said the tests were in lp/code/tests/test_branchmergeproposal
User5: User4 that seems very small?
User2: User5 maybe it's lp/code/model/tests/test_branchmergeproposal.py
User5: User2 aha!
User7: User2 tellurium == codehost only?
User7: User2 and to what revno were you expecting? is currtently 9250
User2: User7 oh, 9250 is probably enough
User2: User7 what is strawberry at?
User7: cool - that's tellurium btw; haven't checked the slaves yet
User7: snap...
User7: 9233. OLD as the hills... or me.
User2: User7 can you bully 9250 or newer on there?
User7: User2 tho if I update that - is that going to break if the DB isn't in sync?
User2: User7 it doesn't talk to the database any more
User7: okis
User4: User7 do you have notes on why edge is so far out of date?
User7: User4 there's a bug somewhere.....
User4: User7 heh
User7: ha. No. I mean a real one. not generally/amusing. hahaha :-D
User3: There's the CSS bug, the lp.testing bug... any others?
User7: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/559128 <== basically auto updates are suspended as they've been breaking. We even a "Be Ware!" note doing CP's that they too can go spectacularly south.
User6: Bug #559128: edge rollouts broke CSS <Launchpad Foundations:In Progress by flacoste> <https://launchpad.net/bugs/559128>
User4: plumbing emergency
User4: AFK
User5: ok, and how do I run a single test ?
User5: User3 ^
User3: User5 I generally use bin/test -cvvt nameoftest
User5: User3 what generates bin/py ?
User3: User5 buildout, probably. Just run make.
User5: ok
User5: whats the magic to fix
User5: Getting distribution for 'zope.testing==3.9.4'.
User5: Error: Couldn't find a distribution for 'zope.testing==3.9.4'.
User5: ?
User3: Update your download-cache.
User3: Or just rocketfuel-get
User5: make update-your-download-cache ?
User3: bzr up ~/launchpad/lp-sourcedeps/download-cache
User3: rocketfuel-get will do that for you, though.
User5: User3 hmm, not finding the test
User3: User5 Which test, and what are you running?
User5: its a single test in a unittest test file, not a doctest
User5: with -ct it seems to find it
User5: I'm running 'testr run -- -ct testname' now
User2: User7 so are the code import slaves on a newer branch yet?
User7: User2 alas no...
User2: User3 i pushed a fix for the serveOverHTTP hang to ~launchpad-committers/launchpad/python2.6
User4: User2 did your emacs change font when upgrading to lucid?
User2: User4 maybe
User2: it certainly has done on upgrade before, don't remember if the lucid upgrade did
User4: User2 because I don't like the current font but I can't remember what I had :(
User2: User4 heh
User2: User4 i seem to be using "DejaVu Sans Mono"
User4: seems good enough I suppose
User4: I don't feel happy with any of them
User7: I used to use one called 'console' for terminals; bu tthe latest version of 'konsole' makes a *horrible* mess of that and a few other fonts; so switched to dejavu sans mono, which is the least worst...
User4: User2 how is your afternoon going?
User2: User4 ok i guess
User4: User2 need to talk or are you chugging along nicely?
User2: User4 still working on the no-hosted-area puller, i deleted a few too many tests
User4: User2 I can't do the qa I wanted to do
User4: heh
User2: User4 argh
User4: I could fix the issue that sinzui assigned to 10.04
User4: for us
User5: User4 User2 for one of you; argh
User5: timeout proposing the branch
User5: https://code.edge.launchpad.net/~lifeless/launchpad/merge
User5: ok got it in on 4th try
User5: https://code.edge.launchpad.net/~lifeless/launchpad/merge/+merge/23521
User3: User2 Only 'slightly horrible'? :P
User3: User5 There seems to be something really really wrong with that diff.
User4: the timeouts suck
User4: I wonder why you are getting those
User3: I guess because you proposed to db-devel rather than devel.
User5: User3 I proposed to the thing lp suggested I should
User3: Yeah, the development focus is not actually the focus of development, in Launchpad's case
User5: wasn't there a thread to fix that ?
User4: well...
User4: the development focus is the default stacking target
User2: User3 there's no monkey patching
User2: at least
User3: User2 True.
User2: User5 the timeouts are probably because the branch was still being scanned
User2: (yay branchrevision!)
User3: User9 You appear to have fixed bug #526826; you might want to assign it to yourself and mark it as Fix Committed.
User6: Bug #526826: test_ftparchive failing on Lucid <soyuz-publish> <tech-debt> <Soyuz:Triaged> <https://launchpad.net/bugs/526826>
User3: User2 Is BranchRevision going to become less stupid some time soon?
User3: User8 Is trusted.sql tested anywhere?
User2: User3 hopefully
User8: make schema ensures it is syntactically correct.
User8: Individual bits get tested when those application layer bits get tested.
User8: such as the triggers etc.
User8: check constraints etc. not so much, as they are just a safety net (the application needs to handle validation nicely, rather than have the database throw back exceptions to the end user)
User5: User3 should I repropose against devel ?
User3: User5 Probably. Otherwise ec2 land won't DTRT, and the diff will make people scream.
User5: why doesn't devel turn up in the web form ?
User5: done
User5: User4 your trivial fix is done, I think.
User2: grr fricking slow acceptance tests :(
User3: User2 Slow, or hanging?
User2: just slow
User4: do we have official redacted text?
User3: User4 Private teams used <hidden>.
User3: But there's already special text for redacted import URLs, isn't there?
 * thumper shrugs
User4: User2 I'll do the merge conflict if you like
User2: User4 that'd be cool
User4: fix submitted
User4: what was the solution to SFTPError: Garbage packet received ?
User4: using ec2 land
User4: _ensure_ec2test_user_has_keys is in the stacktrace
User2: User4 merge devel
User2: User4 but that was quite a while ago, is this an old branch?
User4: User2 no, newly created from devel about an hour ago
User2: or if it's branched of production-devel i guess you can run ./utilities/ec2 from devel;
User4: ah
User2: User4 don't know then
User4: no it isn't
User4: it uses the LCA from devel and production-devel
 * thumper pqm submits
User5: User4 I know its near EOD; if you could tell me if I did roughly the right thing - even if its not a full review - that would be awesome
User4: which url?
User5: https://code.edge.launchpad.net/~lifeless/launchpad/merge/+merge/23523
User5: User4 ^
User4: commented
User5: thanks!
User5: User4 why does mergeFailed not dequeue ?
 * mwhudson submits no-hosted-area to ec2 test -- should be good for a laugh -- and EOWs
User5: User4 in fact, I'm deleting mergeFailed, because its cleaner.
User5: User4 if its not deleted, dequeue has to be a special case
User5: User4 if you haven't eod'd, I've replied.
User9: User3 thanks, didn't realize there was a bug for that
User10: good morning
User3: User9 What were the 2.6 pygpgme failures that you mentioned earlier?
User3: I don't see any.
User8: Who prepared out Python2.5 packages? I'm missing setuptools and easy_install isn't there either for the quick fix (catch-22 there I guess...)
User9: User3 I submitted a branch for bumping to the existing r49 of the sourcecode/pygpgme branch, assuming it would have been tested before it got landed there. But actually, it breaks things
User9: User8 that would be me and wgrant
User3: User9 Ah, so it didn't actually land?
User8: Is it quick to get the packages made (eg. copying the packages from hardy into the PPA)? Or should I just install setuptools from source?
User8: c/hardy/karmic
User3: Why do you need setuptools?
User8: I need to generate 2.5 eggs
User3: Ah.
User3: A no-change rebuild should do it, I think.
User9: User3 specifically, pygpgme seems to be returning unicode strings instead of byte strings all over the place now, leading to doctests puking all over the place - plus some other fails :-(
User3: Lucid?
User8: Yup
User3: User9 Hm. Damn. We'd best fix that.
User8: So I just find the Karmic package and copy it into the Launchpad PPA, with the rebuild option selected?
User9: User8 no!
User9: why do we need this, btw?
User8: There is an open bug in distutils, the end result being we need to use eggs of pytz rather than tarballs.
User8: I can't generate a new 2.5 egg of pytz at the moment because setuptools isn't installed on my box.
User9: oh, yes. I really need to got back to upstream on that
User8: I can just install from source if it isn't a quick fix
User9: s/distutils/setuptools and distribute/
User8: Yer - first part of installing from source is working out what needs to be installed ;)
User9: User8 Lucid has switched from setuptools to distribute. Installing setuptools on lucid would be.... potentially chaotic
User9: I wonder if I should do a no-change upload of distribute to the ppa
User3: Worth a try.
User8: It won't be a dependency, so it won't make anything explode
User3: Uh... well...
User3: I might point out that *buildout* is involved here.
User9: :-)
 * stub lunches
 * maxb lpnochange distribute
User8: ta. I'll give that a spin when I'm back
User11: Morning
User2: User12 so, how do i use subunit?
User2: i have this test results mail and i'd like to have a list of failing tests
User12: mwhudson, you could open it in tribunal, use subunit2pyunit or use subunit-filter
User12: mwhudson, is there no list of failing tests in the email?
User2: not that i can see
User2: also subunit-filter is breaking
User12: mwhudson, can you please forward me the email?
User12: mwhudson, how is it breaking?
User2: User12 http://pastebin.ubuntu.com/415439/
User2: User12 mail sent
User9: User2 Your hackery regarding test hangs under python2.6 is.... intriguing :-)
User9: I am rather mystified, as I can't pin down the underlying cause of the problem appearing
User2: User9 yeah, i couldn't reproduce the problem in a small example
User2: but my socket knowledge has bit rotted a bit
User9: oh, really? I did
User9: h=HTTPServer(); h.start_server(); h.stop_server(); h._http_thread.join()  <<<<HANG
User2: the issue is that in 2.5 socket.shutdown() gets called when another thread is in accept, and accept returns when this happens
User2: User9 i meant 'just using the socket module'
User2: in 2.6 the other thread is in select(), not accept()
User9: oh, really? I didn't pursue the shutdown angle. I went straight to 'why doesn't closing the socket kick it out of select()'
User9: and the answer seems to be 'Python has a LAAAAME wrapper that attempts to implement platform-agnostic socket.dup() which doesn't actually close the socket'
User9: I think I might have a try at simplifying your changes - I postulate that adding a server._http_thread._sock.close() cleanup will do the job
User2: ah
User2: i wasn't very awake when i was digging :)
User9: hmm. oh gosh. I wonder if the very fact we're inside a select on the socket is what keeps alive the refcount which stops the socket being closed
User2: well, in 2.5 we're in an accept() when stop_server is called
User13: Morning, all.
User12: mwhudson, I'm going to spend a little time digging around your subunit failure
 * jml is off the phone now
User12: mwhudson, fuck
User12: mwhudson, I think this is a \r\n problme
User14: User2 thanks for fixing my fsckup
User14: User12 I responded to some of your questions on the pools LEP
User12: bigjools, thanks.
User14: some new mockups done
User14: well, changed
User3: User14 How inappropriate would it be to rSP bugs when closing them from a changelog?
User14: User3 rSP?
User9: removeSecurityProxy?
User3: User14 removeSecurityProxy. It's already pretty much done in the normal case, where process-upload.py uses a permissive security policy.
User14: right
User14: my initial reaction is: nofuckingwayareyoumad?
User14: :)
User14: the right approach is to get the security model correct
User14: rSP is just papering over turd-smeared walls
User3: What is 'correct' here?
User3: It is.
User3: But switching to a permissive policy during a webapp request sounds approximately infinitely worse.
User14: we need to define "correct"
User9: When do you close bugs from a changelog from a webapp request?
User14: queue accept
User9: ah
User14: User13 around?
User9: Why doesn't the accepting user have permission to close the bug anyway?
User3: User9 The bug is private.
User9: hmm
User13: bigjools, hi
User14: hey deryck, can you check scrollback and give your opinion please?
User3: (a similar thing can also now happen if a public bug is Won't Fix, although I believe all of ~ubuntu-archive has sufficient privs to avoid that problem)
 * deryck looks
User14: User13 just 3 mins back
User3: Bug 564491
User6: Bug #564491: Cannot accept package which closes inaccessible bug <oops> <queue-page> <ui> <Soyuz:Triaged> <https://launchpad.net/bugs/564491>
User15: wgrant, User14 Hi!
User15: ;)
User14: guten tag henninge
User13: bigjools, do you mean how do I feel about using  removeSecurityProxy?
User3: Morning henninge.
User15: Is bug 539185 valid?
User6: Bug #539185: BuildBase implementation for build farm jobs <buildfarm> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/539185>
User3: That sounds like a noodles question.
User14: User13 that's a no-no AFAIAC, but it would be good to see if you have any good ideas on how to deal with this situation "properly"
User3: AIUI he is reworking that all.
User14: he is
User15: so I should not touch that atm?
User3: User15 That sounds sane.
User14: User15 have a chat with him, you might be able to help :)
User15: or that ;-)
User12: mwhudson, it's definitely a \r\n bug
User16: bigjools, naughty, naughty boy... ;)
User14: User16 I know, I suck
User14: assumptions are the mother of all fuckups, as they say
User16: bigjools, but sure, if it needs helping, and henninge can do it while making sure it works for us as well, excellent :)
User13: bigjools, what is the user when trying to close a bug from a changelog?  I'm afraid I need to understand better.
User15: User16 I would have expected nothing else
User14: User16 well that would be beneficial to us all, yes :)
User15: User17 Are you out to lunch? ;-)
User14: User13 let's have a call
User14: I can explain
User3: Speaking of the buildfarm... can we have a launchpad-buildfarm project so we can stop randomly assigning bugs between Code, Soyuz and Translations?
User16: henninge, you are in the same country, just drop by him
User15: ;-)
User14: User3 yeah, I'll try and set that up
User15: User16 true, it's my turn. Last time he came by me.
User16: wgrant, no, let's just assign them all to soyuz
User13: bigjools, sure.  skype or mumble?
 * danilos puts on an innocent smile
User14: User13 given that I wasn't around when mumble accounts got handed out, skype :)
User14: hopefully they'll be openid today or monday
 * wgrant hastily uses launchpadlib to reassign all soyuz-build bugs to danilos.
User14: lol
User16: heh
User13: bigjools, give me 2 minutes to close something here.
User16: that API thing was a big mistake!
User17: User15 nope, I'm here.
User14: User13 ok call when you're ready, you can guess my skype id ...
User14: mmm can't see any ash in the sky yet
User3: I wonder when air traffic will resume.
User14: this evening they say
User14: but they said this morning, yesterday :)
User3: Qantas here is saying Sunday.
User14: depends on where each airline's planes are located I guess, they need to catch up
User3: I guess, yeah.
User3: But it probably depends more on what the eruption does...
User12: we really should have "components" in the tracker
User12: mwhudson, I guess you're beyond caring right now, but I can't track down why the critical '\r' character is being dropped
User17: henninge, danilos, User3 bug updated (well, commented).
User3: User12 +inf
User16: noodles775, thanks
User15: User17 great thanks!
 * henninge goes to lunch now ...
User15: ;-)
User16: noodles775, that does sound great as well, thanks :)
User17: np.
 * noodles775 follows henninge
User9: User17 Hi. Has the ec2 of lp:~maxb/launchpad/use-pygpgme-r49 hung, or has it died and just not sent me an email?
User18: User9 it would've died, that's what's happened to me twice now so I expect it to not be a coincidence
User12: I thought we fixed that issue
User18: It's a different one probably
User18: I have no problems with other branches, just with this specific one
User5: User12 you might like, in your 'advocating apis sub-hat', the recent commits to lp:pqm
User12: lifeless, thanks for the heads up
User9: User18 Hi. I have run into test failures when I try to update pygpgme to the update you landed on the sourcecode branch. Do you know anything about that?
User14: User12 do you know much about twisted's TimeoutMixin?
User12: bigjools, I know a bit.
User14: we appear to see it completely ignoring the timeout that was set
User12: bigjools, show me the cdoe
User14: and an exception ensues
User14: User12 top of lib/lp/buildmaster/manager.py
User12: bigjools, today's stable recent enough?
User14: oh yes
User12: bigjools, can you show me the error too?
User14: yup one sec
User14: User12 http://pastebin.ubuntu.com/415490/
User13: Could someone please subscribe me to bug 441039?
User6: Bug #441039: Ubuntu One crashed on launch <apport-crash> <i386> <ubuntuone-karmic> <Android's Fortune:Invalid> <NULL Project:Invalid> <Ubuntu One Client:Fix Released by dobey> <ubuntuone-client (Ubuntu):Fix Committed by dobey> <https://launchpad.net/bugs/441039>
User12: bigjools, have you reproduced this on a development environment?
User14: User12 hahaha
User12: bigjools, I can help you write a unit test for this.
User14: sounds good
User12: bigjools, couple more questions first though
User14: User3 you have a local builder setup, can you re-produce this?
User12: bigjools, what is config.builddmaster.socket_timeout set to on production?
User3: User14 It doesn't actually need a daniel29@example.net. But I will try it.
User3: Oh, wait, that one does.
User3: Not the one I thought.
User14: it's where it ignores the timeout when scanning it
 * wgrant firewalls.
User14: User12 180
User12: bigjools, how did you figure that out?
User14: User12 I looked in the production configs
User12: bigjools, are we sure they are the ones being used in the deployment with this error?
User14: User12 reasonably, why?
User14: I'll get lamont to check
User12: bigjools, well, if it's not set, or if it's set to a value higher than the system socket timeout, then no amount of debugging will help.
User14: system is like an hour, I thought?
User14: but maybe it's not the default any morie
User12: bigjools, well, as far as I can tell from that log you pasted, it is timing out roughly three minutes after the process starts
User14: which is 180 seconds as the config says
User12: right.
User13: adeuring, can you subscribe me to bug 441039 please?
User6: Bug #441039: Ubuntu One crashed on launch <apport-crash> <i386> <ubuntuone-karmic> <Android's Fortune:Invalid> <NULL Project:Invalid> <Ubuntu One Client:Fix Released by dobey> <ubuntuone-client (Ubuntu):Fix Committed by dobey> <https://launchpad.net/bugs/441039>
User12: so what's happening that differs from what you are expecting?
User14: I wasn't expecting a traceback
User12: deryck, done
User3: User14 So, I get a connection timeout after 20 seconds here. It looks like the dev timeout should be 40 seconds.
User14: but I don't know twistd well enough to know what it's supposed to do when the timeout is set
User10: User13 "The following errors were encountered:Deryck Hodge has already been subscribed"
User14: User12 is there a callback from the setTimeout() call if it does time out?
User13: adeuring, ah, ok.  someone must have already.  thanks.
User14: User3 yeah dev is 40. ummm wtf
User12: bigjools, yeah, timeoutConnection
User12: bigjools, http://paste.ubuntu.com/415493/
User14: User12 which is not defined in the buildd-manager ....
User14: ho ho ho
User12: bigjools, the default implementation disconnects
User14: User12 ok I'm trying to work out what effect that would have on b-m
User12: http://twistedmatrix.com/documents/current/api/twisted.protocols.policies.TimeoutMixin.html fwiw
User3: User14 To be clear, I get a traceback like that, except with a normal socket exception.
User14: ok
User14: so I can't figure out if that's normal for the timeout that was set or if something else causes it
User12: well, it's not an error that TimeoutMixin generates
User12: but it's reasonable to expect xmlrpclib to explode if its socket is disconnected from under it.
User14: we don't get the "Scanning failed with:" message so it's not hitting the scanFailed callback
User14: I would have expected twisted to catch that and call the right callback in the Deferred
User12: hmm
User14: which is what we normally see
User14: so I still wonder if it's a twistd bug
User12: bigjools, what kind of object is "self.slave" in the traceback you pasted
User14: heh
User12: bigjools, nvm
User14: this stuff is like black magic
User5: its not that clean
User14: ha
User14: I know that it's an xmlrpc thingy but I can't find exactly where it's instantiated
User12: bigjools, because it looks like it's using a non-Twisted xmlrpc client
User14: because this code is so obtuse
User12: I also can't find where the 'echo' method that is called in the traceback is defined
User14: xmlrpclib.ServerProxy
User14: in there ^
User12: no it's not.
User12: ServerProxy has magic __getattr__ evil
User12: but where's the server-side implementation that it's trying to call?
User14: User12 see lib/canonical/buildd/slave.py
User14: it's magic, like I sid
User14: said
User12: ok.
User14: :)
User14: XMLRPCBuildDSlave specifically
User12: bigjools, so the thing is, we hate transparent RPC in Twisted
User14: s/in Twisted//
User12: bigjools, well, we hate it enough to do something about it
User3: This is why we have the even more transparent launchpadlib...?
User12: bigjools, your hatred is like a candle to our sun
User14: rofl
User12: self.slave.callRemote('echo', ...) is the way it's normally done in Twisted
User17: User9 hey. I only received emails about trivial-bad-httpcaller..., py2.6-warnings and tolerate-lucid-apt... It was definitely running as I checked it's status when you asked yesterday, so I'd say it died :/
User5: User3 launchpadlib makes me think of the movie Mystery Men
User14: User12 basically this won;t work then until we make both sides use twisted's xmlrpc gubbins?
User9: User17 ok. no need to resumbit, it has further problems that need addressing anyway.
User12: bigjools, so, I _think_ what's happening here is that the twisted client with the timeout mixin is doing its job and pulling the rug out from under the non-twisted client
User12: bigjools, but that's only a guess
User12: bigjools, yeah, I think so.
User14: User12 I concur with you
User3: Why does the slave need to?
User14: User12 I wonder if there's anything we can do on the client side to handle this better in the timeout callback
User14: where the slave == the server side (confusing I know)
User12: bigjools, you mean, as a work-around?
User14: yes
User12: bigjools, nothing leaps to mind
User14: I have no desire to start re-writing the xmlrpc innards :(
User12: bigjools, and as for adding another obscure workaround to this code... the phrase "house of cards" leaps to mind.
User14: :)
User14: the stuff at the top of lib/lp/buildmaster/model/builder.py also deals with timeouts
User12: bigjools, you could wrap the non-twisted xmlrpc calls in a try/except block -- that'd be your quickest way out, I think.
User12: bigjools, yeah, that'll be what's triggering the error you see.
 * bigjools idly flicks the bottom card away
User12: gah
User12: I have no idea how this code works at all
User14: is there anyone here in Sao Paulo who's prepared to do some kidnapping?
User3: We really really need to refactor things a bit so RecordingSlave can DIE.
User3: Hahah.
User3: I know how it mostly works... but it's mostly evil.
User14: every time I think I understand it, I promptly get confused again
User14: RecordingSlave was a quick way to avoid doing what I hinted at earlier - re-writing the xmlrpc gubbins to be twisted
User12: hmm
User12: are there any bugs filed about cleaning it up?
User3: It's not *that* hard. Just sometimes you'll get a RecordingSlave, sometimes you'll get a BuilderSlave, and sometimes it depends on who called this particular bit of code this time. And then you get fake responses which are fricking confusing and aaarrrrgh.
User12: I might add them to my someday/maybe list.
User14: it's basically proxying what's in BuilderSlave
User12: you know
User12: after I fix subunit, land the ssh server changes, make pretty graphs for bugs and make our test suite runnable more than once on a computer
User14: can I have a flying car
User3: Only if it's asynchronous.
 * jml returns bigjools a Deferred that will fire a flying car on success
User14: ha
User12: actually, maybe before the runnable-more-than-once thing
User14: User12 I wonder if we just need to remove that timeout in builder.py and wait for the twisted one to work instead
User14: it seems like they will compete
User12: bigjools, they'll definitely compete
User12: zoinks
User14: User12 this is not using twisted xmlrpc at all, RecordingSlave is just setting up a load of Deferreds
User14: the mixin seems like a total waste of time?
User12: bigjools, well, dispatchBuild and checkDispatch use it
User12: I think
User8: User9 That distribute package just seems to have 2.6 stuff in it. No 2.5.
User9: ugh
User12: unfortunately, I've got a bucket load of stuff to do today. :(
User9: It must not respond to the general python version selections presented in python-defaults and python-support
User9: I am about to lunch. I'll take a look afterwards
User14: User12 no worries, thanks for your time
User12: bigjools, np. my pleasure.
User14: User12 I think I can throw an extra catch in here to get that timeout but I've no idea how to test it!
User19: allenap, Do you have time for a call about bug 530113?
User6: Bug #530113: Information on a bugwatch error is obscure <story-reliable-bug-syncing> <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/530113>
User20: User19 In about 3 minutes (ordering computer parts, woot).
User19: allenap, Cool, I'll sit around on t'mumble.
User12: bigjools, I know how to test timeouts with pure Twisted systems
User12: boo yah
User12: the first of my four SSH branches just passed ec2 test
User12: (they failed yesterday because devel was broken)
User14: User12 I'll think of something hackish no doubt
User14: User12 I apologise for breaking devel
User12: bigjools, np. the real problem is the system that allows a simple human error to screw up other people's workflows
User14: aye
User14: User12 if you have a sec can you look at lib/lp/buildmaster/model/builder.py -  updateBuilderStatus()
User14: and tell me what else it needs to catch to get that timeout?
User12: bigjools, that ought to be it...
User14: you'd think :/
User12: bigjools, easy enough to figure out though. Make a TimeoutHTTPConnection work-a-like, give it a low timeout, see what error gets raised
User14: right
User14: shame it's not in the log :(
User14: mmm this makes me hungry
User12: I think it's more that socket module has crap errors.
User13: yay, gnome-bugs bugs are updating!
User13: gmb, error message question for bug watch update activity....
User13: gmb, see bug 44082
User6: Bug #44082: GNOME Panel icons (on right side) move apparently randomly on session start in some situations <gloam> <qa-hardy-desktop> <qa-jaunty-desktop> <qa-karmic-desktop> <GNOME Panel:Fix Released> <One Hundred Paper Cuts:Confirmed for ryan.maki> <gnome-panel (Ubuntu):Confirmed for desktop-bugs> <gnome-panel (Ubuntu Hardy):Triaged by desktop-bugs> <https://launchpad.net/bugs/44082>
User19: Urr.
User21: deryck, its more like "oh no! mail flood!"
User13: nigelb, for you. :-)  For me, it's "oh happy day, praise be to the internet almighty." :-)
User21: deryck, haha.
User19: deryck, Whatsyerquestion?
User13: gmb, so expand the first task row.... that error there?  Is that from bug watch activity log?  It seems out of date, since the bug is in fact updated now.
User19: deryck, It's not from the activity log, no, it's from the last_error_type property of BugWatch. Let me just have a poke around and see if I can find out what's oging on.
User13: gmb, cool, thanks.  I want this expandable form to go away.  It causes no end of grief since it gets out of sync with the rest of the app.
User9: Is there any mechanism by which a ~launchpad-pqm sourcecode branch can be push --overwritten?
User19: deryck, User20 AHAHAHAHA
User13: oh, do tell
User19: deryck, So, that error is a real error. The status sync succeeded but the backlinking failed (and our errors aren't smart enough to handle that yet; will fix that this pm).
User19: deryck, OOPS-1567CCW1152
User19: (Another win for BWA, though)
User13: indeed!  This logging is working out well.
User19: deryck, User20 Looks like we don't have permission to backlink on the remote system.
User20: User19 Oh, jolly good.
User19: allenap, I think it might be because we (used to?) backlink for every watch, whether the LP bug was relevant or not.
User19: So the gnome-bugs admins disabled backlinking.
User20: User19 That sounds right.
User19: allenap, Is there an easy way to disable backlinking on our end to prevent the errors or is it not fine-grained enough for that? ICR.
User20: User19 Not fine-grained enough.
User20: User19 We could quite easily disable all back-linking by removing the ISupportsBackLinking interface from BugzillaAPI, but that could/would affect other trackers.
User19: allenap, Yeah.
User19: allenap, Though, hang on, if we can't sync comments why are we trying to backlink anyway?
User19: That seems like huffing on the bong pipe to me.
 * allenap looks
User20: User19 It's not conditional on sync_comments.
User19: allenap, I think it should be. That would solve this problem for a start.
User20: User19 The condition, in full, is: (bug not dupe and linked to at least one bugtask and supports back-linking)
User16: noodles775, hi, do you have any suggestions how could I QA https://code.edge.launchpad.net/~jtv/launchpad/bug-553077/+merge/22599?
 * noodles775 looks
User19: allenap, Right. Do you think that sync_comments should be used there too? I mean, I know it's a misnomer if we do that, but...
User8: User9 np. I worked around by installing from pypi. I then uninstalled it after generating the eggs, as buildout exploded ( https://bugs.edge.launchpad.net/launchpad-foundations/+bug/564680 ). I'm not sure if your package will cause the same issue, but maybe it is best not bothering (I can survive, and we should be on 2.6 sometime soonish)
User6: Bug #564680: buildout explodes when distribute is installed <Launchpad Foundations:New> <https://launchpad.net/bugs/564680>
User17: User16 nope.
User17: User16 I mean, if the buildd's are still running, it's effectively qa'd.
User3: But that code is not on production at the moment.
User17: User16 so, ensuring dogfo..
User16: noodles775, so, how do I get that tested on dogfood?
User17: User16 so, ensuring the buildd's used by dogfood are updated, and then that it still works.
User17: User16 as long as we ensure the buildd's are updated, we can just push a build through on df.
 * noodles775 checks with lamont to get the buildd's updated on dogfood.
User16: noodles775, cool, thanks (I totally haven't done anything about this, so I don't even know where to start)
User22: danilos, looks like you found a bug in paramiko: line 163, "str(data[:chunk])" should be unicode(data...)?
User22: danilos, don't know why it would start failing now though
User16: mars, it might not be recent, I haven't used ec2 for a while
User22: hey, paramiko uses Launchpad! :)  https://edge.launchpad.net/paramiko
User16: mars, I can file a bug, if you think that's the thing to do :)
User22: danilos, not sure if switching to unicode would help.  I just tried an experiement in ipython, and reproduced that error.  It is the str() call.
User22: str(u'MÄris')  raises the error
User16: mars, right, it'd be worth seeing if it makes sense to .encode('utf-8') before passing it out to paramiko: it might be that unicode there won't help either and it'll barf later if no UTF-8 locale is used
User22: danilos, worth a shot.  Looking at the docs, and sftp should support utf-8 encoding for filenames
User16: mars, well, UTF-8 wasn't called UTF for filenames for nothing on Plan9 I think :)
User22: hehe
User16: anyway, me goes get some late lunch, will be back later for a short while
User22: rocketfuel-branch is the wrong command to use when making a one-line patch :/
User17: User23 do you have time in the next 0.5hr for a brief call regarding the style.css work?
User17: (no problem if not).
User23: I do
User22: gary_poster, have a sec to review a one line patch?  https://code.edge.launchpad.net/~mars/launchpad/fix-ec2-email-encoding/+merge/23557
User24: mars, looking
User24: mars, I'm guessing you have empirical proof it works? :-)
User22: gary_poster, just a traceback from danilos sent to lp-dev.  This same fix is present throughout the ec2test.py file.
User22: gary_poster, I have have danilos try it out before submitting to PQM
User24: mars, alright.  I have vague memories of pain with UTF in email headers and the stdlib email packages, but even if things don't get better, this shouldn't make anything worse (and hopefully it is a fix, of course!).  Yes, would be happy if someone gave it a try before merging.  Will approve with that condition, since you already have it planned.
User22: gary_poster, ok, thanks.  I'll send a mail to danilos saying as much.
User24: cool
User24: approved
User14: jml`: still busy?
User14: jml`: ok should be quick :)
User14: see lib/lp/buildmaster/model/builder.py "resumeSlaveHost"
User14: versus
User14: lib/lp/buildmaster/manager.py "checkResume"
User14: the former is called when b-m starts up and reports the stdout/stderr if it fails
User14: the latter is asynchronous and I don't know how to do the same
 * jml` pulls up the code
User14: is stdout/err buried in response somewhere?
User14: I don't think so
User14: unless one's the recording slave and the other's the BuilderSlave
User14: who knows :/
User14: actually yes checkResume is dealing with a RecordingSlave
User14: resume is when we restart the VM
User14: it ssh'es to the slave and calls a script
User14: the slave's host I should say
User14: in config
User14: ppa_reset_command or something
User14: anyway
User14: I need to get at the output of running that - it uses run_process_with_timeout
User14: ok let's keep it simple
User14: checkResume gets a response from run_process_with_timeout
User14: if the process failed I want its stdout/err
User14: \o/
User14: :/
User14: if it wasn't Friday afternoon I would be sprawled on my desk in despair
User14: I guess it was used because it has a timeout
User14: and the name seemed right
User14: *shrug*
User14: is this nearer the mark? http://twistedmatrix.com/documents/current/api/twisted.internet.process.Process.html
User14: doesn't seem to handle timeouts
User14: you da man
User14: lookng
User14: ok
User14: jml`: looks great, thanks
User14: "improved"? :)
User14: right
User14: don't think it is, the script works in ~6 seconds
User14: ah ok, don't really need that sophistication here I think
User14: it's an ssh - it can die die die :)
User14: yer
 * bigjools hacks
User18: User9 still there?
User14: jml`: in timeoutConnection you're using error and it's not defined
 * bigjools loves the vim pyflakes plugin
User11: Guten nacht
User9: User18 back now
launchpad-dev
Ubuntu IRC