User1: hey Chipaca, good morning!
User1: 7096 needs a second review, should be an easy one
User2: User1 looking
User1: ta
User3: mmh
User1: User3 mmh? (and good morning :)
User3: I have vague memories that the ! would go in front , but I'm probably misremembering, the notes says after
User3: also the shell would scream if in front
User3: so I'm misremembering for sure
User1: User3 oh, indeed
User3: also in the context of the unset work, we shouldn't forget we decided to fix snap set a.x=... a=... by sorting
User1: User3 good point, I wonder if we captured this in one of the trello cards
User2: User3 what kind of sorting?
User2: User4 User1 do you know, offhand, how to pause the live cdimage before it boots into the actual live system? wanting to set SNAPD_DEBUG :)
User4: not sure you can ... but there should be a console on tty4
User4: (so you can do your edits and restart all involved bits)
User2: User4 the thing i'm wanting to see is the seeding, and i'm not fast enough
User1: User2 uh, thats a tricky one. I wonder if you could break into initramfs
User4: yeah, thats a possibility but also a lot of time consuming work
User4: (manually assmbling the rootfs, chrooting and editing etc)
User2: eugh
User2: I've asked laney, maybe they have a trick for it
User4: try break=bottom
User4: on the kernel commandline
User1: yeah, break=bottom was what I was thinking, then tweak and exit the shell and hope for the best
User2: trying that
User4: (there should be a way to enter this at the very start)
User2: break=bottom and then echo SNAPD_DEBUG=1 >> /root/etc/environment worked
User1: User2 \o/
User2: need to step out for a bit, will bbiab
User3: User2 by depth
User1: 6878 (health status in info/list has two +1, I will merge it, I left one bikesheed comment though, but its fine to ignore that)
User3: User1 it's in the card https://trello.com/c/yxC3omC9/245-snap-unset-support
User3: but not sure it was done yet (haven't looked at the PRs)
User1: User3 excellent, thank you!
User1: User3 not done yet in any of the PRs I have seen so far
User1: fwiw 7098 and 7102 should be easy wins
User3: we are also missing snapctl unset
User1: User3 good catch
User3: anyway I added actual todos to the card
User1: User3 thanks for adding it to the card!
User1: fwiw - I closed 5915 now (netplan apply) now that we have #7107
 * mvo really misses mup btw
User3: User1 thank you
User1: 7100 is now also green - its "funny" one, it requires that we update gorilla too
User3: it needs Chipaca to look at it, then I will look at it as well
User5: sergiusens, hi, would you consider merging https://github.com/sergiusens/snapcraft-preload/pull/32? we'd need it for postgres, which tries to create /dev/shm/PostgreSQL.XXXXX files
User2: User1 wrt 7100, why not have the prereq helpers also take context?
User2: User1 they're called from a do* handler so you've got a tomb so you've got a natural context already
User1: User2 aha, let me check that
User2: User1 i mean, if they break all the tests in the world, maybe it's worth leaving them for later :-)
User2: but otherwise, ...?
User1: User2 let me explore that
User1: User2 I'm a bit slow (sry!) do you have an example helper name that could take the context for me? not sure I know what you have in mind
User2: User1 installOneBaseOrRequired
User2: User1 via installPrereqs
User1: User2 aha, yes
User1: User2 I can do that, however I think the reason its not done (now) is that the client-user-agent will only be set (in this PR) in the SnapAction store API call. once the change is created the client-user-agent is no longer available in the change. we could fix that but its more work and the win seems to be small. or am I missing something?
User1: User2 we would have to persist the client-user-agent somewhere in the change if we want to keep it and use it in the handlers
User2: User1 hm...
User2: ah so it's not in the tomb at all
User2: sary, missed that disconnect again
User2: User1 in any case +1
User2: User3 does request-serial in classic hit the actual store?
User1: User2 \o/ thank you!
User1: 7098 is green and has one +1 already and is smallâ¦
User2: User1 it's got two +1's though
User3: User2 yes
User2: User3 that's taking ~30s
User2: User3 which is the same as seeding takes
User2: on this machine
User2: User3 seems slow for a single request
User2: Â¯\_(ã)_/Â¯
User3: that is interesting
User2: User3 https://snapforum.s3.amazonaws.com/original/2X/5/55ca16db9b44fe934424c27999c3386948b4fb4e.png
User1: User2 heh :)
User1: User2 we are down to 51!
User2: User3 it's prepare-serial-request, not submit-serial-request, that takes that long though
User3: User2 ah, but we don't split retries
User3: so a bit unclear
User3: if that's one request or many retries
User2: User3 you can ask
User2: User3 you can ask me, even
User2: or you can get the iso and look for yerself :-p
User3: prepare-serial-request
User3: that is even weirder
User2: anyway i need to go comb my hair (just one of them) and get ready for the show
User3: are we doing or changed something silly
User3: we ask only a nonce from the store there
User3: I need to dig/look around a bit
User2: User3 you ok to repro this on your own, or should i get the logs somewhere
User2: User3 instructions here: https://forum.snapcraft.io/t/snap-seeding-is-slow-racy/12310
User3: User2 well, I'm not promising to repro it
User3: I'm promosing to understand what we need to look
User3: more precisely
User2: User3 fair enough
User2: i look forward to learning more :-D
User2: but now i need to run
User1: User2 I added a comment to 7083 that might be worth doing (not sure though, depends on the plans with 7092)
User2: User1 ok, i'll take a look in a bit
User2: afk from irc but github will work :-)
User3: I left some comments there
User3: User2 it takes ~200ms to get a nonce from the store, so something else is going on
User1: 7102 is another easy win
User1: User8 snapd PR#7031 (re-enable the system PATH generator) is fixed now, right? or do we need some initramfs fix too (I have some vague memories that this was discussed but don't know the conclusion)
User1: User8 once you  +1 it I will merge it
User4: sil2100, could we get this into all gadgets (core16 and 18) soon ? https://github.com/snapcore/pi3-gadget/pull/28
User4: (i asked ondra for a signoff too, to get more eyes on it, but in general someone should really monitor the pi firmware and push regular updates to the gadgets, they often fix over/under vlotage issues, overheating etc)
User6: User4 sil2100 i thought we just landed the same thing
User4: cwayne, well, i couldnt make any of our official images boot on the CM3 on my desk ... neither core16 nor core18 (though i dont look much at 18 anyway given the quality)
User4: CM3+
User4: sorry
User4: using my universal pi-kiosk snap with the latest firmware (also for pi4 support) boots it fine though
User4: otherwise it always hangs on the rainbow splashscreen (meaning the rpi bootloader itself dies before it can load anything)
User6: User4 did you try with a very recent image? think the updated gadget landed in stable ~5hrs ago
User6: worked for 18 for us
User4: i tried both, 16 and 18 from the current dir on cdimage
User7: Hi! one of my snap failed to publish because I added a new hook, is that normal will have to go through a review or is that something that needs changing in my yaml ?
User7: the warning was "unknown hooks in meta/hooks: 'disconnect-plug-services-content' lint-snap-v2_unknown_hook"
User4: i also dont get why we still have separate images for pi2/3/cm3 and dont use the universal gadget, i thought we moved to that
User5: jdstrand, hi, wrt /run/uuidd/request, do you think it makes sense to file a bug for an interface request? I see it's currently available in openvswitch-support
User7: roadmr ^
User8: User1  you are correct. Needs https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1814355 tasks for initramfs-tools done.
User8: let me comment on the PR
User1: User8 thank you
User9: User5 last we talked in Lyons, blake, et al all felt that the snap should be adjusted to use /proc/sys/kernel/random/uuid, which is standard and in the default template). there is nothing that will guarantee that /run/uuidd/request will be there on all distros. that said, it seems systemd is providing it and snapd requires systemd (and 14.04 is no longer supported by snapd)
User5: jdstrand maybe we can use snap layouts for that
User5: jdstrand, we don't explicitly use that socket, libuuid does that
User9: User5 well, sure, but why use libuuid? :)
User5: jdstrand, actually we can't use templates as the kernel one is just a file you cat
User5: jdstrand, because it's somewhat standard? IDK we don't use it explicitly :)
User5: I mean it's not a direct dependency
User5: a lot of stuff use that
 * jdstrand wonders why it doesn't use the kernel interface rather than uuidd
 * cachio afk
User9: (when the kernel supports it)
User5: jdstrand, dunno, portability?
User5: I mean, sure libuuid could use that file if it's there
User9: right
User9: anyway, I've taken a todo to investigate this for the default template
User9: I suspect systemd will be looking at that file
User5: jdstrand, the python uuid uses os.urandom() afaics
User10: User4 that should be released already
User10: User4 the 'master' branch is out-of-date and not used
User10: User4 I actually need to just remove it
User1: User11 just fyi, I'm poking at the tests in your netplan-apply pr right now
User10: User4 there are 16 and 18 branches, and both have the latest uboot bumps for the pi3
User4: sil2100, well, none of the images i tried on the CM3+ booted
User4: sil2100, this is not u-boot ... this is the raspberry pi firmware
User4: sil2100, it doesnt even get to a point where it would load u-boot, hangs in teh step before
User4: (on the ranbow colored splash screen (which shoudl really be disabled, not sure why thats on in the official gadgets)
User10: User4 ok, I see what's the problem - you are right, the core16 gadgets didn't get the firmware bump
User10: User4 since we only bumped the uboot version there
User4: right
User10: User4 let me do that then
User4: thanks !!
User10: User4 but the 18 ones are bumped if anything
User10: ETOOMANYGADGETREPOS
User4: wried
User10: Sorry about that!
User4: *weird even
User4: sil2100, i definitely tried the rpi3 UC18 images from /current
User4: (as well as the (totally pointless) cm3 ones ... why do we build them ?)
User4: sil2100, could it be that the fix hasnt made it to /current yet ? (i can surely try /pending too if needed)
User10: hm hm hm, it looks correct, manifest shows having the latest pi gadget, and from what I see in the gadget's build log the right boot-firmware tag has been cloned
User10: And I thought waveform checked if it's the right firmware version before submitting the PR
User11: User1 thanks
User1: User11 I pushed some bits, if you mind I can also push to a separate PR
User11: no, it's fine you can push to the same PR, thanks!
User1: User11 mostly suggestions, probably need a critical eye from samuele as well but should make things a bit easier
User11: User1 ack, thanks for that
User1: User11 its mostly there now, something in the TestYesNetplanApply test is unhappy still, it hangs here
User1: User11 not sure why, probably some mocking missing for something
User11: yeah I've seen it hang a few times I think it has something to do with locks
User11: err I've seen it hang in a few iterations I've done
User1: User11 yeah, its probably not my changes :)
User1: User11 aha, locks! good catch, let me check those
User11: the version I pushed up shouldn't hang, but maybe your changes are moving back to what I tried originally that was hanging
 * mvo nods
User4: sil2100, looking at https://github.com/snapcore/pi-gadget/blob/18-armhf/snapcraft.yaml i see a tag from feb 2019 ... perhaps thats still not new enough
User10: User4 maybe, but waveform selected that tag exactly because of cm3+ support
User10: Let me poke him
User10: User4 anyway I'd expect it to work as we use the exact same firmware tag for disco and eoan classic raspi3 images, and I'm sure those work on the cm3+
User10: (also, per Dave's original commit message: https://github.com/snapcore/pi-gadget/commit/b8a16cb1e1d9a2c0419da8fef0bffc12e3c914fb )
User4: very weird ...
User10: I asked him to test again
User10: Maybe there's something weird going on somewhere
User10: I'll wait for this to be resolved before I update the 16 gadgets, since I'd like it all to use the same firmware
User7: When a new hook is added to a snap package (e.g. interface connection/disconnection hook), does that need to be approved by the store guys ?
User11: User7 I'm not sure that the store review tools have been updated to take into account interface connection/disconnection plugs
User11: jdstrand and roadmr should be able to help you with that when they get time
User4: sil2100, well https://github.com/snapcore/pi-gadget/commits/18-armhf shows that all travis checks have failed ... not sure if that tree is gated by travis though
User4: (for both last commits)
User10: User4 well, it passed on the PRs
User10: User4 (when you check the failed travis checks, those are issues with git clone and I also experienced that during the snap builds)
User10: And even locally
User10: There was a period of time where the default git repo url of uboot that we use was not usable
User10: Anyway, basically the travis checks are build-checks, so if LP was able to build the end snap, that means it passed all the tests
User7: ijohnson thanks, I'll wait
User4: ok ... as long as the sync to LP isnt blocked by these travis checks i guess it is fine
User4: just looked like a potential reason for not having the change in these images
User10: User4 I checked the snap build logs, and I saw that the git clone of the firmware branch printed "Note: checking out '98997f363e3683ead4f50c37902169248628303a'.", which is a commit from feb 2019
User4: ok
User4: very strange then
User10: User6 hey! Do you guys have cm3+'s in the lab?
User4: they do
User12: User7 can you show me (snapcraft.yaml) what you added?
User6: They're on the way sil2100
User6: plars has one at home
User10: User6 awesome \o/
User13: User10 have a new image I need to try?
User7: roadmr https://github.com/deskconn/deskconn-server/blob/master/snap/snapcraft.yaml
User12: User7 hm this looks reasonable; I think the best way would be for you to change what you need, upload the snap, and the review tools will tell you if it needs manual approval
User10: User13 could you check the current one? Since Dave seems to be AFK - just so that someone else also tries booting it http://cdimage.ubuntu.com/ubuntu-core/18/current/ <-
User12: User7 typically what we do is approve that in general, and subsequent uploads won't need the same thing
User13: User10 will do
User10: User13 thank you!
User4: plars, i couldnt get that one to boot on my CM3+ ...
User7: roadmr https://dashboard.snapcraft.io/snaps/deskconn/revisions/84/
User4: would be good to have another datapoint
User7: I think revision 82 is the one that needs approval
User10: This will be some good input, since I at least hope he did test that gadget change before submitting the PR
User10: (Dave that is)
User6: User10 we had tried an image with a gadget using Dave's pr and were able to boot core18 but not core16
User10: User6 yeah, the core16 is expected, because of uh, my mistake
User10: Too many repos of those gadgets
User10: User6 but the 18 gadget booted fine on the cm3+?
User6: User13 ^
User6: Yeah that was my understanding
User13: yes, still setting up to try this one though - I'm also trying to boot it with a regular cm3 in the lab
User10: The regular ones should be good, since I saw those being tested in the automated testing
User4: yeah, very unlikely that they could have regressed
User13: User4 did that rpiboot from edge work for you at least? seems to be working on my cm3+
User13: it's writing the image now
User4: sil2100, oh ... note that i tested http://cdimage.ubuntu.com/ubuntu-core/18/stable/current/  is that different from http://cdimage.ubuntu.com/ubuntu-core/18/current/ ?
User10: User4 yes!
User4: oh man
User4: can we clean up that directory mess ?
User10: User4 we don't daily-build stable images, only during point releases
User10: User4 you need to check the timestamps, we only daily build edge and beta images right now
User4: oh my ... ok
User10: User4 stable is like the stable ones we want to brand and 'release', which only happens on every point release
User10: ;)
User4: then sorry for the noise ... i followed some download link (that i have lost now :( ... )
User4: from our main website somewhere
User10: http://cdimage.ubuntu.com/ubuntu-core/18/current/ are for edge and http://cdimage.ubuntu.com/ubuntu-core/18/beta/current/ are for beta - and I guess for a daily stable image you'd have to build one locally with ubuntu-image --channel=stable
User10: It might be a decent idea to build some stable images before point releases
User10: But doing it daily wouldn't make much sense, there's not enough velocity there
User4: well, yeah, dail stable (or daily in general) is nonsense anyway ... the images shoudl simply be built change-triggered
User10: So we'd probably switch some stable images built on every stable snap change, but that's still not there
User10: Yeah, that's the plan
User10: But we still don't have that
User10: plars, User6 issue has been resolved, sorry for the commotion o/
User4: http://cdimage.ubuntu.com/ubuntu-core/18/stable/current/ doesnt really suggest it is anything thats daily though ... if i navigate from the toplevel of cdimage i indeed want the "stable " release ... and then go to the most current image of that by using the "current" link
User4: so even if i had not follwed a download link originally,m i would have ended up in the exact same place to get the latest stable image
User10: Well, the header of the stable images mentions 'Ubuntu Core 18', where others explicitly say 'Daily Builds' - but indeed, we might want to make this a bit more intuitive
User10: User4 well, this is the 'latest stable image' ;)
User4: right ... but there are no headers at http://cdimage.ubuntu.com/ubuntu-core/18/ :)
User10: User4 as mentioned, we will only promote to 'current stable' on point releases, so people following download links will always get only the image we blessed during point release preparation
User10: True
User10: cdimage is a confusing place
User10: ;p
User4: and when i finally enter the site i have to search for the right image between the gazillio pi images on that page (is there any reason why we do not have one pi image per arch /armhf/amd64 ?)
User13: User10 this boots on both my cm3 and cm3+, but it has edge snaps for everything. Is that expected?
User13: ah, just saw the backlog
 * ogra hands plars some https://www.youtube.com/watch?v=7nqcL0mjMjw
User4: :)
User13: hah
User13: User10 so we are waiting on a different image to test now I guess?
User10: User13 no no, it's all good - I mean, I'll be pushing new cm3 16 gadgets out soon, but other than that we're good
User13: ack
User4: sil2100, but in all seriousness, why do we have pi2, pi3 and cm3 images there ? supposedly all using the universal pi gadget and the same kernel, so in the end they are all the same image
User4: this smells like a massive waste of built/test time and server space
User4: (i originally developed the universal pi gadget to exactly not have three images anymore :) )
User10: User4 well, it is a waste, yes, that was some leftover of the 16 times, no reason for it - guess when the first 18 models have been created and signed we simply cloned what we had for 16
User10: So we got pi2, pi3 and cm3 models, thus a separate image for each model
User10: oh well
User4: sil2100, yeah, for 16 we couldnt change ... i had hoped for 18 it would though
User4: i guess i'll send some expensive tea to bribe xnox to make sure core20 gets better here ;)
User10: User4 a missed opportunity I guess! There was a bit too much going on so I also didn't notice it's unneeded until it was already all released ;)
User4: yeah
User1: User11 please check netplan-apply - should be mostly good now, feel free to fix anything I missed and there is also a spread test missing iirc
User11: User1 thanks I just saw your push I was waiting to push up my spread test until you were done
User4: well, who knows ... i'm not sure we can actually support the pi4 with the same kernel snaps in the end ... perhaps it was a clever thing to keep them separate
User4: until we can go to a new enough kernel that supports all of them in core20
User1: User11 yeah, I think I'm finished with what I wanted to do :) feel free to tweak/adjust as needed, its your PR afterall, I feel slightly bad for being so pushy in it :/
User1: User11 but it was also fun so I couldn't stop myself
User11: User1 no problem, I appreciate the help in the tests that was what I was blocked on so you have unblocked me :-)
User1: User11 great! happy to help and to be fair the testing was a bit delicate (but hopefulyl we can make things better/easier :)
User11: +1
User3: 7083 needs a bit more work, there or in a follow up
User3: it has 2 +1 but I should probably look at it as well
User3: I commented about the work
User9: hmm, om26er is gone. ijohnson> fyi, so long as the hook is known to the review-tools at all, then there is no problem. if a new hook is added to snapd, then the review-tools need to learn it
User3: User11 I was discussing with mvo, a smallish thing that we might need sooner rather than later, related to remodeling is this:  https://forum.snapcraft.io/t/remodeling-to-a-new-model/10477/2
User11: User9 does the store tools know how to handle interface hooks? IIUC, those hooks could be named anything in the patterns shown on https://docs.snapcraft.io/interface-hooks
User11: User3 looking
User9: User3 I'm not up on remodeling and I don't see anything in that forum that references hooks (am I blind?)
User3: User9 not related to the discussion, but ijohnson has progress quite a bit on his current task, so I was discussing with mvo what he can pick up next
User3: *progressed
User3: *can or could
User9: oh, heh, I copied the wrong link. durr
User11: User3 okay this seems straight forward, but actually like jdstrand I'm not familiar with the current plan on remodeling :-)
User11: User3 is that forum post all we have for the plan for remodeling ? I see you also have quite a few PR's open with remodelling label
User3: User11 we progressed quite a bit
User3: that forum topic is mostly concerened with reregistration, switching model
User3: User11 I'm close to eod but we can chat more tomorrow
User9: User11 the tools already handle those, yes
User9: User11 (https://docs.snapcraft.io/interface-hooks)
User11: User3 okay sure I will try to schedule a meeting tomorrow to discuss more
User11: User9 hmm okay well it looked like using an interface hook was the problem that om26er was running into
User9: User11 ah, the tools know about connect-*, prepare-* but not disconnect-*
User9: I see that r82 uses disconnect-plug-...
 * jdstrand adjusts tools
User9: User12 hi! can you pull 20190716-1700UTC? not urgent. fixes om2er's issue (cc ijohnson)
User12: User9 sure thing, I'll queue it up
snappy
Ubuntu IRC