Beta Server Office Hours/Minutes/2011-06-02

Category:Beta Server Office Hours - Minutes
This meeting was held on June 02nd, 2011
== Agenda ==
* Introducing Alain.
=== Updates ===
* Second Life Server (main channel)
** This has Kelly's "maint-server" 
** https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_Server/11#11.05.17.230139
* BlueSteel RC Channel
** This channel contains the "friendship" project.
** Basically updates for the People API.
** Modified the capabilities of the People API to handle adding/removing friendships.
** Included are other infrastructural changes that are generic in nature.
** Fix to allow services to start properly after deployment of new code.
** https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_BlueSteel/11#11.05.31.231546
* LeTigre RC Channel
** This is now Kelly's "mono2-performance" project.
*** Third time is the charm?
** https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_LeTigre/11#11.05.31.231545
* Magnum RC Channel
** This is "mesh-prep"
** lLCastRay() is disabled.
** Physics changes.
** https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_Magnum/11#11.05.31.231501
* Snack RC Channel
**Snack is closed.
=== Upcoming Stuff ===
* Group Chat pile-on this afternoon.
** https://wiki.secondlife.com/wiki/GroupChatTest
= Transcript =
Transcript of Oskar Linden's Beta Server Office Hour for June 02nd, 2011:
* [15:00] Kallista Destiny: Ave Oskar, , laguri te salutant.
* [15:00] Coyot Linden: It's cuckoo for Coco-Puffs!
* [15:01] Gecko Linden: thee Oskar
* [15:01] Oskar Linden: hey everyone
* [15:01] Alain Linden: hi
* [15:01] Kallista Destiny: good grief, we seem to be outnumberd by Lindens... No biteing.
* [15:01] Gooden Uggla: hiya oskar
* [15:01] Coyot Linden: Ave, Salutes te morituri
* [15:01] ANSI Soderstrom: heya
* [15:01] Jacky Scharf: Good morning everybody. 
* [15:02] Kallista Destiny: *chuckles*
* [15:02] Fancy Greeter: Maestro Linden has arrived! (Or, returned?)
* [15:02] Kallista Destiny: O'im not dead yet.
* [15:02] Coyot Linden: You got better?
* [15:02] Oskar Linden: my machine is lagging out a bit here
* [15:03] Rex Cronon: greetings all
* [15:03] Homeless: expected: behavior
* [15:03] Kallista Destiny: actually I feel much better.
* [15:03] Oskar Linden: ansi can you turn of fyour pacifier please :-)
* [15:04] Oskar Linden: hmm. my machine is unresponsive
* [15:04] Coyot Linden: Did you say please?
* [15:04] Homeless: is: it plugged in?
* [15:05] Oskar Linden: it is plugged in for sure
* [15:05] Gooden Uggla: did you clear cache between versions?
* [15:05] Oskar Linden: :-p
* [15:05] Homeless: power: on?
* [15:05] NielArcher: are: you using the correct mouse and keyboard? ;-)
* [15:05] Oskar Linden: ok. there we go. feels a bit better now
* [15:05] Gearsawe Stonecutter: have SL installed?
* [15:05] Rex Cronon: wireless keyboard?
* [15:05] Oskar Linden: here is the agenda link
* [15:05] Oskar Linden: https://wiki.secondlife.com/wiki/Server_Beta_User_Group
* [15:05] Oskar Linden: First up is introducing Alain Linden
* [15:06] Kallista Destiny: Did you turn it off and on again?
* [15:06] Oskar Linden: Alain, tell these good folks hi
* [15:06] Alain Linden: hi
* [15:06] Alain Linden: ;-)
* [15:06] Rex Cronon: hi
* [15:06] NielArcher: hi: Alain
* [15:06] Oskar Linden: who are you and waht do you do?
* [15:06] Homeless: hi: Alain
* [15:06] Draconis Neurocam: Hello Alain
* [15:06] Gooden Uggla: *waves*
* [15:06] Jacky Scharf: Hello Alain. 
* [15:06] Sigma Avro: bonjour Alan
* [15:07] Coyot Linden: no voice
* [15:07] Kallista Destiny: Welcome to the nuthouse, Alain
* [15:07] Alain Linden: heh
* [15:07] Oskar Linden: alain has been working on the group chat fixes
* [15:07] Oskar Linden: he's been doing lots of performance testing with our bots
* [15:07] Kallista Destiny: Oh!! Alain we love you, I hope.
* [15:08] ANSI Soderstrom: bad bay
* [15:08] Kandi Luminos: semaphore flags and a gesture for each?
* [15:08] TriloByte Zanzibar: excellent, we're looking forward to that
* [15:08] ANSI Soderstrom: boy
* [15:08] Alain Linden: well, you can sort of love me
* [15:08] Jacky Scharf: Uhhh? And when will the fixes be on Agni?
* [15:08] Alain Linden: we made group chat somewhat better.
* [15:08] Latif Khalifa: "somewhat"?
* [15:08] Oskar Linden: somewhat.
* [15:08] Alain Linden: as in better, but we don't know if it is betterer enough
* [15:08] Jacky Scharf: wtf? .
* [15:08] TriloByte Zanzibar: I suspect that in order to make it completely better, you need to build a new system
* [15:09] Kallista Destiny: for some value of better.
* [15:09] Oskar Linden: the pile-on test today will be a good test
* [15:09] Guardian: i: will be taking part!
* [15:09] Sigma Avro: morse could be used
* [15:09] Coyot Linden: niiiiiiiice
* [15:09] Oskar Linden: we'll talk more about the particulars later in the meeting
* [15:09] Latif Khalifa: do you think a few dozen users here can be a representative test?
* [15:10] Alain Linden: oskar, would it be worth going over the changes we're testing and how it improves chat?
* [15:10] Jacky Scharf: What's the duration of the test today?
* [15:10] Coyot Linden: ---...---
* [15:10] Oskar Linden: I think we'll get more then a few dozen
* [15:10] Sigma Avro: lol Coyot
* [15:10] Oskar Linden: who is beeping btw?
* [15:10] Kallista Destiny: Broken
* [15:10] Coyot Linden: SMILE
* [15:10] TriloByte Zanzibar: yeah, even with the short notice I think we'll get decent turnout
* [15:10] Coyot Linden: he's doing Morse code
* [15:10] Latif Khalifa: I would love to hear about the changes Alain
* [15:10] Oskar Linden: I think we'll get a decent turnout
* [15:10] Kandi Luminos: the new Morse Chat System?
* [15:10] Kallista Destiny: very slowly..
* [15:10] TriloByte Zanzibar: let's get through the agenda, so we can hear more from alain :)
* [15:10] Coyot Linden: err, could you kill that?
* [15:11] Coyot Linden: please?
* [15:11] Sigma Avro: done
* [15:11] Gecko Linden: ahhhhhhh
* [15:11] Coyot Linden: tyvm
* [15:11] Oskar Linden: that was very annoying
* [15:11] Homeless: yes: very
* [15:11] Oskar Linden: alright then
* [15:11] Oskar Linden: next up on the agenda
* [15:11] Sigma Avro: no more than chat lag could be !
* [15:11] Gecko Linden: but it was a little entertaining.
* [15:11] Oskar Linden: this has been a much better week for release
* [15:11] Oskar Linden: things are stable
* [15:11] Coyot Linden: \o/ \o/
* [15:11] Oskar Linden: perhaps even happy
* [15:12] Coyot Linden: coyot is happy!
* [15:12] Oskar Linden: coyot's attitude has improved dramatically
* [15:12] Coyot Linden: I'm not horizontal anymore
* [15:12] Oskar Linden: we put Kelly's maint-server to the main channel
* [15:12] TriloByte Zanzibar: so far, so good there
* [15:12] Oskar Linden: the roll had some hiccups since it had been a couple weeks for some of those regions to be restarted
* [15:13] Oskar Linden: it was taking a long time for them to save their simstates
* [15:13] Coyot Linden: just required more patience than usual
* [15:13] Sigma Avro: lol
* [15:13] Homeless: a: couple of weeks is a long time
* [15:13] Oskar Linden: but release got it donw
* [15:13] Oskar Linden: BlueSteel has the people api changes again
* [15:13] Oskar Linden: this will probably get the nod for promotion next week
* [15:14] Latif Khalifa: now homeless is ticking lol
* [15:14] Homeless: good: mood today
* [15:14] Coyot Linden: +1 on BlueSteel
* [15:14] Oskar Linden: LeTigre got the new and updated mono2-performance code for the 3rd time
* [15:14] Maestro Linden: first time!
* [15:14] Latif Khalifa: 3rd time the charm!
* [15:14] TriloByte Zanzibar: the equestrian-friendly version?
* [15:15] Maestro Linden: or was it the 3rd time? I forgot the first two
* [15:15] Gooden Uggla: poor le tigre is next to explode...
* [15:15] Coyot Linden: tick
* [15:15] Latif Khalifa: Meastre, did you ship it with 3 script deserialization a frame?
* [15:15] Coyot Linden: tick
* [15:15] Oskar Linden: I think it is the 3rd time on an RC
* [15:15] Coyot Linden: tick
* [15:15] Maestro Linden: yes Latif
* [15:15] Latif Khalifa: seemed like a sensible default
* [15:16] Sigma Avro: and snack is dead
* [15:16] Latif Khalifa: Homeless, that's really annoying
* [15:16] Coyot Linden: yay!
* [15:16] Rex Cronon: what is annoying?
* [15:16] Maestro Linden: yes, though unfortunately there's a performance issue still when an avatar with a few hundred/thousand scripts enters a mono2performance region..which we're working on today (SCR-88)
* [15:16] Oskar Linden: I think Homeless is ticking
* [15:16] Sigma Avro: thks ANSI
* [15:16] Oskar Linden: thanks for the update on that maestro
* [15:16] Homeless: well,: i AM a clock!
* [15:17] Coyot Linden: one that needs to be cleaned? :P
* [15:17] Oskar Linden: Magnum had some funniness last week
* [15:17] Gecko Linden: go easy
* [15:17] Oskar Linden: that's the mesh-prep branch
* [15:17] Oskar Linden: and that branch had some creative ideas about prim counting
* [15:17] Oskar Linden: I think we got those issues found and fixed
* [15:18] TriloByte Zanzibar: coming up short on prim count reports?
* [15:18] TriloByte Zanzibar: oh good
* [15:18] Sigma AvroSigma: Avro still loading meshes with no name. Wander what inventory issues it will drive
* [15:18] Oskar Linden: but in the meantime some parcel owners took advantage of the fact that the prim counts were off and overdid their allotment
* [15:18] Oskar Linden: then things got returned
* [15:18] Gecko Linden: "whoops"
* [15:18] TriloByte Zanzibar: some may not have even realized
* [15:18] Latif Khalifa: or weren't aware of the bug
* [15:18] Oskar Linden: true
* [15:19] Oskar Linden: that's all fixed up now right maestro?
* [15:19] Gecko Linden: ouch
* [15:19] Maestro Linden: yes, as far as I know, SVC-7004 is fixed. Multiple people have confirmed it so far..
* [15:19] Flame of: Jira: [#SVC-7004
* [15:19] Oskar Linden: there were some other issues tied to vwr 2.7, but those weren't Magnum issues
* [15:19] Latif Khalifa: we will se a lof of fun with this one mesh deploys. simple resizing of the mesh object changes prim counts.
* [15:20] Oskar Linden: changes what it costs. it shouldn't change parcel allotment
* [15:20] Kallista Destiny: ??
* [15:20] Oskar Linden: this bug was changing parcel prims allowed
* [15:20] Latif Khalifa: well changes the number of free prims you have, that's what i mean
* [15:20] Oskar Linden: gotcha
* [15:20] Latif Khalifa: same thing
* [15:20] Maestro Linden: Oh, so I have a question for Magnum parcel owners - have you had any objects using HTTP-in that got a URL_REQUEST_DENIED response after wednesday's roll?
* [15:21] Kallista Destiny: Well the number of primes shouldn;t change GReatly should they?
* [15:21] Maestro Linden: I was investigating SCR-86 a bit, and I'm not sure if it was a freak incident or what
* [15:21] Flame of: Jira: [#SCR-86
* [15:21] Kallista Destiny: Prims
* [15:21] Sigma Avro: will be a speciall tag on parcell for meshes in additin to prims, or meshes will be converted in prims ?
* [15:21] Oskar Linden: right kallista. the number alloted should not change. The number of free/unused prims in your allotment can change with meshses of different size
* [15:21] Kandi Luminos: Oskar, we have a couple of Magnum sims in our estate, and I hadn't noticed that
* [15:22] Kandi Luminos: or rather... Maestro
* [15:22] Oskar Linden: I don't have the details on mesh -> prim accounting
* [15:22] Maestro Linden: ah, thanks Kandi
* [15:22] Oskar Linden: those details will be made available closer to the ship date for mesh
* [15:22] Sigma Avro: ok
* [15:22] Oskar Linden: speaking of, I think a mesh blog post went out
* [15:22] Kallista Destiny: Yes that should be 'interesting'.
* [15:23] Oskar Linden: 
* [15:23] Latif Khalifa: yes, says mesh on RC sims on Agni in July
* [15:23] TriloByte Zanzibar: yeah, excellent news
* [15:23] Latif Khalifa: so we have a month for "details"
* [15:23] Oskar Linden: :-)
* [15:24] Oskar Linden: We've used up all our Snack's
* [15:24] Gooden Uggla: i'll be back in november :)
* [15:24] Oskar Linden: Snack is not being used on AGNI this week
* [15:24] Oskar Linden: or we might have it open for something else
* [15:24] Oskar Linden: I forget heh
* [15:24] Coyot Linden: /o\
* [15:24] Oskar Linden: we attempted to test out a fix for the TIMEWARP issue
* [15:24] TriloByte Zanzibar: will it be RC'd on the existing RC channels, or will they have their own rollout schedule?
* [15:25] Kallista Destiny: foo
* [15:25] Oskar Linden: I don't think our fix worked
* [15:25] Oskar Linden: so we're still looking into it
* [15:25] Latif Khalifa: I hope Lindens will reconsider changing the model so that object scale does not factor in number of prims meshes are charged against parcel limits. Someone resizing their mesh tree in the garden should not make them run out of free prims. It violates the principle of least astonishment.
* [15:25] Oskar Linden: I don't know TriloByte, but I'll find out
* [15:25] Maestro Linden: It's possible that the one Snack sim host with 'TIMEWARP' issues has a hardware problem, also. That's my hope, at least...
* [15:26] Oskar Linden: that's a discussion for the mesh usergroup Latif
* [15:26] Latif Khalifa: I'm only expressing my hope :)
* [15:26] Oskar Linden: :-)
* [15:26] Oskar Linden: anything upcoming to mention Maestro?
* [15:27] Maestro Linden: So, mesh-prep changes on Magnum broke some vehicles
* [15:27] Latif Khalifa: I was glad to see the bug in the mesh-prep that has broken third party import tools has been fixed.
* [15:27] Maestro Linden: such as the Monorail in Toyosu (the cars get stuck in places)
* [15:27] Maestro Linden: Falcon has a 'unified fix' which will hopefully fix these vehicles while not breaking other content
* [15:28] Homeless: how: are the horses doing?
* [15:28] Latif Khalifa: lol
* [15:28] Maestro Linden: so far they're back to normal, thanks for asking :P
* [15:28] Coyot Linden: Frau Blucher!
* [15:28] Psi Merlin: I think I got to know that Snck server - but I still feel the script pauses I saw may have had another factor involved - SCR-88 could explain some of it.
* [15:28] Homeless: lol: 
* [15:28] Flame of: Jira: [#SCR-88
* [15:28] Kallista Destiny: I expect what you'll see is that an object with 1 physician prim costing 6 might suddeny cost 12 as it expands in size. I think that people will quickly understand this.
* [15:28] Maestro Linden: we'll hopefully have a new build with falcon's fix in the "DRTSIM-52" channel of aditi later today or tomorrow
* [15:29] Oskar Linden: well those are our standard updates.
* [15:29] Oskar Linden: we can talk about the group chat changes now
* [15:29] Rex Cronon: this increase of nr of prims that is because of size. will it affect all prims or just meshes?
* [15:30] Kallista Destiny: Mesh only
* [15:30] Oskar Linden: does this test plan look good to you all?
* [15:30] Oskar Linden: https://wiki.secondlife.com/wiki/GroupChatTest
* [15:30] Oskar Linden: I tried to keep it simple and straightforward
* [15:30] Oskar Linden: we have 2 islands set up and need people on different islands chatting in group chat
* [15:31] Kallista Destiny: Looks fun.
* [15:31] Sigma Avro: Oh, before, just a question :one of my residents observed on magnum that prims resisting to return occured apparently due to roll restarta just a point befoer : could be prims resisting to return due to roll restarts ?
* [15:31] Homeless: unfortunately: I have to leave after OH, 6pk dinner time here
* [15:31] Homeless: 5pm: 
* [15:31] Oskar Linden: ok homeless
* [15:31] Homeless: 6pm: (typonese attack)
* [15:31] Jacky Scharf: Group chat across region borders or every group on each region?
* [15:31] Oskar Linden: does anyone have any questions about the steps in that wiki page?
* [15:31] TriloByte Zanzibar: unless dinner involves a 6pk :)
* [15:32] Gecko Linden: happens to me too Homeless
* [15:32] Rex Cronon: how many people know about the test toda?
* [15:32] Oskar Linden: we will be using the Second Life Beta group to group chat in
* [15:32] Rex Cronon: today*
* [15:32] Jacky Scharf: Hmmm ... need an invite.
* [15:32] Kallista Destiny: Linden's are not allowed to speak typonese.
* [15:32] Oskar Linden: word of the test spread pretty quickly
* [15:32] Draconis Neurocam: would conferences not tied to a specific group also load the server as well, or is it just for the second life beta group?
* [15:32] TriloByte Zanzibar: not enough, Rex, but we should still get a decent turnout
* [15:32] Oskar Linden: that's something else we need Draconis
* [15:32] Jacky Scharf: Oh, I'm there. :-D
* [15:32] Oskar Linden: I didn't want to confuse things on that page
* [15:32] Homeless: especially: when writing code
* [15:32] Techwolf Lupindo: What wiki page? (I forgot the URL)
* [15:33] Latif Khalifa: https://wiki.secondlife.com/wiki/GroupChatTest
* [15:33] Rex Cronon: why didn't u send a group notice on main?
* [15:33] Oskar Linden: if you guys want to open your own conferences to chat in that is a needed test too
* [15:33] Gearsawe Stonecutter: would logging on an alt help?
* [15:33] Oskar Linden: I sent word out to the SL Beta group and AWG on AGNI
* [15:33] Oskar Linden: then asked to have people pass it on
* [15:33] Oskar Linden: it's gotten some good coverage
* [15:34] Kallista Destiny: I passed ti a group of a few hundred
* [15:34] Oskar Linden: our CEO even found the link and tweeted it. that kinda surprised me.
* [15:34] Oskar Linden: you can bring on alts
* [15:34] Oskar Linden: or bots
* [15:34] Oskar Linden: whatever
* [15:34] Pygar Bu: couple of us have posted on Plurk and Twitter a few times too
* [15:34] Kandi Luminos: It's been passed to the RLV support group... I'll get it out to a few thousand more right away
* [15:34] Oskar Linden: it would be good if some of you can create small conferences to chat in
* [15:34] Latif Khalifa: Perhaps Alain can outline the changes made. My understanding about grouop chat problems was that it needed to lookup presence data on each chat participant for each line of chat sent. It is therefore I was asking how effective test on Aditi can be since presence data here is not nearly as big and changing as on Agni.
* [15:34] Gecko Linden: hahaha megga pile on!
* [15:34] Oskar Linden: Draconis, Latif, can you guys do that?
* [15:35] Oskar Linden: Alain, can you outline some of the changes?
* [15:35] Alain Linden: sure
* [15:35] Alain Linden: so I don't know what y'all do or don't know about how group chats work, so I'll give you the readers digest version
* [15:36] Flip Idlemind: Someone should tell the Phoenix/Firestorm support group...that'll get like 100,000 people here
* [15:36] Sigma Avro: if anyone is connected to additi, should ask to beta group to reinforce chat test on aditi
* [15:36] Sigma Avro: *agin aditi
* [15:36] Alain Linden: when someone sends a group chat it goes to your sim which forwards it to a special service
* [15:36] Draconis Neurocam: sure oskar, just need calling cards to add people to them, so people would need to add me
* [15:36] Alain Linden: the service knows who in the group is online and receiving
* [15:36] TriloByte Zanzibar: I did not long ago, with link to the wiki
* [15:37] Rex Cronon: no wonder that lagging sims affect your group chat:(
* [15:37] Sigma Avro: oh, tks Zanzi
* [15:37] Alain Linden: currently, when a message comes in, the group chat service must locate every recipient
* [15:37] TriloByte Zanzibar: also explains why some groups have consistently poorer performance
* [15:37] Alain Linden: that requires a lookup to another service
* [15:38] Alain Linden: then once it knows where each recipient is, the group chat service must send a message to each recipients sim
* [15:38] Alain Linden: so, if you have 250 group members on line and in the group, the group chat service
* [15:38] TriloByte Zanzibar: so it sort of daisy-chains to all the sims that all the avatars in that group (who are online) are on?
* [15:38] Alain Linden: must do 250 look ups followed by 250 sends.
* [15:38] Pygar Bu: whew!
* [15:38] Kallista Destiny: ouch
* [15:39] Sigma Avro: and how is it improved now ?
* [15:39] Rex Cronon: considering that the nr of online user never went above 100k, why don't u just have a table with all users and sim they r in. it might speed thing up a little
* [15:39] Jonathan Yap: the lookups done every time a line of chat is sent?
* [15:39] Gearsawe Stonecutter: geesh. So would people with bad packet loss cause a problem?
* [15:39] Alain Linden: As a first pass at improving things, we wanted to see what we could fix in the present system without changing a lot of code or introducing incompatible changes with current viewers.
* [15:39] Latif Khalifa: these lookups are done synchroneously? i mean it needs to get all the participants before it stars sending?
* [15:39] Alain Linden: That constrains us a lot.
* [15:40] Kallista Destiny: I can imagine
* [15:40] Alain Linden: Not exactly Latif..
* [15:40] TriloByte Zanzibar: cut the 1x cord, working group chat would be a compelling reason to upgrade :)
* [15:40] chan 0: listener: got chat: 1, FruitFox-SpikeyHair, 0a605728-97fb-ef0a-4f4d-68b536e5e618, spiky
* [15:40] Alain Linden: There are two changes we've made that will improve performance.
* [15:40] Maestro Linden: afk a bit
* [15:40] Alain Linden: First, we're caching presence information in the group chat service.
* [15:40] Ryou Debs: ...what if we set up a date some time in the future? Obviously I understand the urgency, but you'd need a lot of people to just blab in a big group chat.
* [15:41] Alain Linden: That should eliminate most of the presence look ups.
* [15:41] TriloByte Zanzibar: that's good
* [15:41] Alain Linden: We figure people don't move from region to region that often during a chat session.
* [15:41] Kallista Destiny: Caching is a good thing
* [15:41] Sigma Avro: mm. second, on changing sim, avatr must send info to chat server ?
* [15:41] Alain Linden: The other change is that we've updated the sim so that we can send one message for multiple recipients.
* [15:42] Techwolf Lupindo: Alain, gridhunters are the exception to that. They tp often to pick up prises.
* [15:42] Latif Khalifa: hmm. risky assumption. so you teleport to another region and miss group chat for the cache timeout time.
* [15:42] Kallista Destiny: Region sends information
* [15:42] Fancy Greeter: Falcon Linden has arrived! (Or, returned?)
* [15:42] Gooden Uggla: really? i have groups i leave open all day to monitor chat...
* [15:42] TriloByte Zanzibar: when a user logs on, will they need to wait until the cache is updated before they start receiving group chats?
* [15:42] Marianne McCannMarianne: McCann often teeps along during group chats
* [15:42] Flip Idlemind: So will that cause problems for people who *do* change regions a lot while group-chatting
* [15:42] Alain Linden: The caching is transparent and if you aren't on the expected sim, the chat service will look you up in the presenece database and send the message again.
* [15:42] Gearsawe Stonecutter: so if you do change to a diffent sim. then what? Or well find out
* [15:43] Latif Khalifa: ok that sounds good
* [15:43] Kallista Destiny: ahhhhhhh, that makes sense
* [15:43] Sigma Avro: ah, so messages will not be received in the same order by all residents !
* [15:43] Kallista Destiny: send the message to the region and if it fails tehn look for the person
* [15:43] Alain Linden: Now by sending one message per sim, we can reduce the number of messages sent so long as there are multiple recipients in a region.
* [15:43] NielArcher: SMILE: they aren't now ;-)
* [15:43] TriloByte Zanzibar: it's a shame the chat couldn't just bypass the sim and go from the central server to the user regardless of region
* [15:43] Gecko Linden: tested that today with small groups, did not notice any lags when moving between sims.
* [15:44] Alain Linden: Unfortunately, our data shows that on average there are 1.1 recipients per region so that isn't going to help as much as we hoped.
* [15:44] Alain Linden: So far our tests show that we've made group chat at least 2X better.
* [15:44] Pygar Bu: any improvement :-)
* [15:44] Sigma Avro: this will introduce another kind of chat lag bug : messages resceived in different order for different recipients 
* [15:45] Jonathan Yap: And what we will be testing tonight are these changes?
* [15:45] Alain Linden: That's not bad, but it remains to be seen if it is enough to really improve your user experience.
* [15:45] TriloByte Zanzibar: you might be surprised
* [15:45] Latif Khalifa: Smile, that's how it works today
* [15:45] Sigma Avro: no Latif
* [15:45] Rex Cronon: we already receive messages in different order
* [15:45] TriloByte Zanzibar: from my experience on the main grid, the groups that have the biggest chat problems are estate-based communities
* [15:45] Sigma Avro: dont think so. Well maybe you are right
* [15:45] Gooden Uggla: if it cuts in half the amount of failed mesages, that's a good thing
* [15:45] Lag Frenzy: no, now we alll get the messages out of order
* [15:45] Alain Linden: By the way, you should know that the bottleneck in chat delivery is the chat service.
* [15:45] TriloByte Zanzibar: which may have higher resident per region numbers... my fingers are crossed
* [15:45] Alain Linden: It isn't the presence service or sims.
* [15:46] Jonathan Yap: Establishing the session has often been an issue for me
* [15:46] Lag Frenzy: only having some of us get them out of order is great
* [15:46] Alain Linden: The game is all about minimizing the number of requests performed by the chat service.
* [15:46] Marianne McCann: Word, Jonathan. "Cannot start chat with "Mental Mentors" blah blah"
* [15:46] Sigma Avro: so, ; dont change sim if you want to chat correctly. It is reasonnable
* [15:46] Latif Khalifa: Alain, there was also some mention of partitioning the chat service based on the group uuid which results in some groups being on "bad" chat servers which are perma-lagged
* [15:47] Alain Linden: So, currently chat groups are partitioned by uuid and that is unchanged.
* [15:47] Rex Cronon: is not that reasonable. it implies that there is no synchronization
* [15:47] Jonathan Yap: How many servers are there?
* [15:47] Alain Linden: So, yeah, if you are on a server that hosts some big talkative groups, your still gonna lose.
* [15:47] Latif Khalifa: 16 last time i heard, one for each starting digit of the uuid
* [15:48] Latif Khalifa: is it possible to add more servers? subpartition?
* [15:48] Jonathan Yap: And they are all real machines, not virtualized?
* [15:48] Alain Linden: We're considering ways to subpartition
* [15:48] Alain Linden: but we haven't yet implemented anything.
* [15:49] Alain Linden: Another thing we've done that doesn't directly affect performance but should be really useful is add more instrumentation to the system
* [15:49] Alain Linden: since we now have a pretty good understanding of lag, we can get out the right numbers to spot it.
* [15:49] Kallista Destiny: nods, you need data to make a good fix
* [15:49] Rex Cronon: why not make group chat a different service, and when somebody logs in they r also logged in the group chat service
* [15:50] Moundsa Mayo: Alain, do chat messages carry a timestamp all the way through to the viewer?
* [15:50] Alain Linden: we don't timestamp messages currently
* [15:50] Moundsa Mayo: k
* [15:50] Flip Idlemind: So for the test today should we stay in 1 place or hop around to different regions
* [15:50] Sigma Avro: yses that would be a good solution: put a time stamp, to avoid chat confusions
* [15:50] Alain Linden: we did do something like that in a test cluster which helped us spot the chat bottleneck.
* [15:50] Kallista Destiny: uses up too much screen space
* [15:51] Moundsa Mayo: Well, then some sort dance would have to occur in viewer chat window, leading to other confusions.
* [15:51] TriloByte Zanzibar: wouldn't it be possible to bypass having to check the sim entirely? that shouldn't have to matter, since regardless of what specific region they're on the resident should receive the message
* [15:51] Kallista Destiny: 15:50:53 test
* [15:51] Coyot Linden: You sent that yesterday, right?
* [15:51] Sigma Avro: (senders time stamp)
* [15:51] Kallista Destiny: Yes...
* [15:51] Oskar Linden: she sent it tomorrow
* [15:51] Oskar Linden: she's in the future
* [15:51] Alain Linden: in theory one could bypass the sim, but the sim isn't really causing much lag.
* [15:51] Gecko Linden: LOTTO!!!
* [15:51] TriloByte Zanzibar: sent tomorrow, it's been tuned up too much! :)
* [15:51] Jonathan Yap: If the message had a timestamp it could be displayed, and the recipient could figure out what was sent when
* [15:52] Latif Khalifa: TriloByte, that's what xmpp was about but it was decided to constain to the solution where viewer will only talk to the sim like now and not directly to the chat service
* [15:52] Coyot Linden: just sayin' ... a timeonly timestamp is not complete
* [15:52] Moundsa Mayo: or it could be interleaved into existing window contetn, with the aforementioned confusion.
* [15:52] Alain Linden: xmpp is a potential long term solution
* [15:52] Sigma Avro: ah, Latif will be hapy now
* [15:52] Alain Linden: recently we've been looking for shorter term improvements
* [15:53] Kallista Destiny: Yes, back in the OLD chat we actuslly had some messages delayed by more than a day
* [15:53] Techwolf Lupindo: Timestamp of seconds past the startime. That how is done internelly in UNIX like systems.
* [15:53] Coyot Linden: usually called the epoch
* [15:53] Coyot Linden: 1/1/1970
* [15:53] Coyot Linden: at midnight
* [15:53] Techwolf Lupindo: Thats it. Epoch. thatnk you coyot
* [15:53] Coyot Linden: yw
* [15:53] Kallista Destiny: 16:00:00 12/31/1969
* [15:53] Oskar Linden: BTW, before the meeting is over, please say Happy Birthday to GECKO LINDEN!
* [15:53] Alain Linden: so, that's the high level view.
* [15:54] TriloByte Zanzibar: happy birthday, Gecko!!
* [15:54] Charlene Trudeau: happy birthday, Gecko!
* [15:54] Coyot Linden: hippo birdie 2 ewe!
* [15:54] Alain Linden: Anything more I can tell you?
* [15:54] Kallista Destiny: Happy Birthday Gecko
* [15:54] Latif Khalifa: sigma, i don't particulary care about xmpp or not as long as we have a working group chat. cacellation of xmpp was bad news because it meant one more delay in fixing the problem
* [15:54] Gecko Linden: hahaha, thanks.
* [15:54] ANSI Soderstrom: happy gecko
* [15:54] WolfPup Lowenhar: happy b-day gecko
* [15:54] Rex Cronon: happy birthday:)
* [15:54] Sigma Avro: jouyeux anniversaire GECKO !!!
* [15:54] TriloByte Zanzibar: interesting insight, alain, thank you
* [15:54] Oskar Linden: yes, Thanks Alain!
* [15:54] Latif Khalifa: Alain, thank you very much for the explanation.
* [15:54] Moundsa Mayo: A very Happy Birthday Gecko, and many more to you!
* [15:54] Jacky Scharf: Ohhh ... Happy birthday dear Gecko 
* [15:54] Gecko Linden: thankee
* [15:54] Alain Linden: BTW, there's a high level commitment to getting group chat working well
* [15:54] Oskar Linden: alright. does anyone have any questions before we shuffle off to the test regions and start chatting?
* [15:54] Pygar Bu: Hippo birdies, gecko!
* [15:54] Coyot Linden: rez day or bday?
* [15:54] Marianne McCann: Hippo Birdies!
* [15:54] Sigma Avro: Well Latif, I discovered you have build your how system, forgot the name
* [15:54] Kallista Destiny: indeed, knowing what the problem is helps us explain ot to the rest of our groups.
* [15:55] Alain Linden: so you can expect to see more changes over the long term
* [15:55] Ryou Debs: ...when you get older, losing your hair...many yeeears from now...do do do do do..
* [15:55] Gecko Linden: bday
* [15:55] WolfPup Lowenhar: we are only to chat in group chat right?
* [15:55] Oskar Linden: yeah
* [15:55] Latif Khalifa: Alain, sorry, but we had commitments for years, including last year SLCC where the CEO said fix by the end of the 2010 :)
* [15:55] Pygar Bu: Hi! Loki!
* [15:55] Oskar Linden: pass this link on to anyone who wants to help out: https://wiki.secondlife.com/wiki/GroupChatTest
* [15:55] Moundsa Mayo: All these architectural details we learn here sure would make some purty block diagrams.
* [15:55] Oskar Linden: tell them to get their prims over to ADITI!
* [15:55] Latif Khalifa: So I'm looking forward to actual fixes, not commitments :)
* [15:55] Marianne McCannMarianne: McCann waves at Loki
* [15:55] Oskar Linden: we have been gathering metrics for before and after graphs
* [15:56] Oskar Linden: maybe we'll have some ready next week
* [15:56] TriloByte Zanzibar: oooh - charts!
* [15:56] Oskar Linden: thanks to everyone for coming today. I'm gonna go head over to the test regions and start chatting
* [15:57] Coyot Linden: cya
* [15:57] Moundsa Mayo: Thanks for your User Group time Oskar, Maestro, Coyot, Alain, Gecko, Falcon! On to the gabfest B^D
* [15:57] Coyot Linden: yw
* [15:57] Latif Khalifa: Do you want us teleort among the regions during the test? Or should we pick one and stay there?
* [15:57] Pygar Bu: yay!
* [15:57] Sigma Avro: Is it better to go on Oatmeal or on GC%20Test ?
* [15:57] Rex Cronon: tc oskar
* [15:57] Oskar Linden: either or latif
* [15:57] Rex Cronon: u mean this is not a test region?
* [15:57] Oskar Linden: pick on sigma
* [15:57] Sigma Avro: Thanks you Oskar
* [15:57] Sigma Avro: oh, lag, o
* [15:57] Oskar Linden: this is not a test region
* [15:58] Marianne McCann: All in the Second Life Beta group?
* [15:58] Oskar Linden: you need to pick a slurl from here: https://wiki.secondlife.com/wiki/GroupChatTest
* [15:58] Latif Khalifa: I am here with 5 bots, you thought I was loud before :P
* [15:58] Jacky Scharf: So thank ya very much ... espacially to the Linden's. 
* [15:58] Techwolf Lupindo: secondlife://Aditi/secondlife/Oatmeal%201/128/128/30 (so I can clink on it)
* [15:58] Marianne McCann: Heading to a GC region
* [15:58] NielArcher: slurls: aren't working for me
* [15:58] Techwolf Lupindo: Hmm...can't tp ther.
* [15:58] Oskar Linden: :-(
* [15:58] Johnnie Carling: no workie
* [15:58] Latif Khalifa: Ima, kill Aditi from it
* [15:58] Sigma Avro: lol tp failed !