Beta Server Office Hours/Minutes/2010-02-11

Category:Beta Server Office Hours - Minutes
This meeting was held on February 11th, 2010
= Agenda =
* Discuss resolution of previous open items.
* Updates
** 1.36.0 is in a pilot roll now.
*** bug list: https://jira.secondlife.com/secure/IssueNavigator.jspa?reset=true&&type=1&pid=10002&version=10568&sorter/field=issuekey&sorter/order=DESC
** 1.36 rolls to AGNI fully week.
*** half on tuesday.
*** next half on wednesday.
** util.aditi rotating out this evening at 8pm PST for ~ 1 hour.
*** Expect degraded service.
** Havok 7 testing is on the calendar
* Releases
** https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_Beta_Server/1.36
* Any other business
** How does the grid do rolling deployments? Direct File Transfer to each server? Faster Way? http://torrentfreak.com/twitter-uses-bittorrent-for-server-deployment-100210
** increased physics time for certain physical objects in 1.36 only. __Zwag
*** http://jira.secondlife.com/browse/SVC-5374
= Resolved Items =
= Meeting Minutes =
* We went through the agenda
* We talked about upcoming server releases
* We talked about Havok 7 testing
* Lil talked about grid region rolling restart procedures
= Transcript =
Transcript of Oskar Linden's Beta Server Office Hour for February 11th, 2010:
* [15:01] Ratchet Xevion: there he is
* [15:01] Dimentox Travanti: accepted your inventory offer.
* [15:01] Kyra Vixen: Greetings, Oskar. ^^
* [15:01] Ratchet Xevion: Hey Oskar lol
* [15:01] Zwagoth Klaar: The man himself.
* [15:01] Oskar Linden: poor morris
* [15:02] Sparacolori+++ /: Crayon Katyusha (trail) glow whispers: Use Mouselook (press 'M') to shoot me.
* [15:02] Zwagoth Klaar: Did it get nuked?
* [15:02] Lares Carter: heya everyone
* [15:02] Sparacolori+++ /: Crayon Katyusha (trail) glow whispers: Choose 'Detach' from my menu to take me off.
* [15:02] sensory Hax: greetings all
* [15:02] Oskar Linden: morris is down
* [15:02] Oskar Linden: and nothing I could do to coax her up wwas working :-(
* [15:03] Zwagoth Klaar: o.O Ouch
* [15:03] Oskar Linden: I didn't have much time to poke the logs
* [15:03] Zwagoth Klaar: It was down when I got on earlier today
* [15:04] Oskar Linden: hrm
* [15:04] Eddi Decosta: woh, nice lag 
* [15:04] Oskar Linden: well ok then
* [15:04] Identity Euler: hi 
* [15:04] Object: Morris: (MATURE) (255232, 256256) is down
* [15:04] Oskar Linden: the agenda is here: https://wiki.secondlife.com/wiki/Beta_Server_Office_Hours#Agenda
* [15:04] Eddi Decosta: Hi hi all 
* [15:04] Jonathan Yap: There are a handful of people at the other meeting area
* [15:05] Oskar Linden: at the border?
* [15:05] Jonathan Yap: Oskar, can you send a grid-wide message to them?
* [15:05] Jonathan Yap: yes, east of here, about 6 ppl
* [15:05] Oskar Linden: good call
* [15:06] ATechwolf Foxclaw: I amade it.
* [15:06] Drew Dwi: the sim is lagging hard
* [15:06] Entering god: mode, level 200
* [15:06] ATechwolf Foxclaw: Hmm...TD of .5 now
* [15:06] Xugu Madison: Did someone fold, bend or spindle the grid?
* [15:06] sensory Hax: nods
* [15:07] Oskar Linden: meeting over here guys 
* [15:07] [[User:xsensory Hax[it>en]|xsensory Hax[it>en]]]: / me nods
* [15:07] Indira Matova: arrgg
* [15:07] Oskar Linden: ok then
* [15:07] Oskar Linden: my comp is laggin hard
* [15:07] Oskar Linden: let's get this thing started
* [15:08] Oskar Linden: Lil will be joining us later
* [15:08] Oskar Linden: the agenda is here: https://wiki.secondlife.com/wiki/Beta_Server_Office_Hours#Agenda
* [15:08] Latif Khalifa: who's killing ahern
* [15:08] Oskar Linden: if anyone has something they want to bring up please add it to the any other business section
* [15:08] Drew Dwi: 10 points if you find the object causing the physics lag?
* [15:08] Oskar Linden: looks at zwag
* [15:09] Zwagoth Klaar: This is a Second Life Production Server 1.34.2.142087
* [15:09] Zwagoth Klaar: sim
* [15:09] Zwagoth Klaar: so it's not me
* [15:09] Drew Dwi: someone rezzed a mono script :P
* [15:09] Oskar Linden: ANSI!!
* [15:09] ANSI Soderstrom: My Group is gone, but cannot re-create while its already exists o.O
* [15:09] ANSI Soderstrom: oskar
* [15:09] Oskar Linden: something ansi owns is tkaing up a massive amoutn of script time
* [15:09] ANSI Soderstrom: still rezzing
* [15:10] Eddi Decosta: Hi Psi! 
* [15:10] ANSI Soderstrom: ok, scripts off
* [15:10] Drew Dwi: smite him all mighty smiter
* [15:10] Eddi Decosta: hi Drew 
* [15:10] Psi Merlin: Hi all
* [15:11] sensory Hax: hi
* [15:11] Oskar Linden: that and ovlatine is also causing a ton of lag
* [15:11] Latif Khalifa: this is one sick sim. someone should take it out of it's misery ;)
* [15:11] ANSI Soderstrom: lag, in SL ? unbelievable
* [15:11] Ovaltine Constantine: Am I causing lag?
* [15:11] Oskar Linden: let me see what else we have
* [15:11] Ovaltine Constantine: I dont think I am
* [15:11] Ovaltine Constantine: But I dont know for sure
* [15:12] [[User:xsensory Hax[it>en]|xsensory Hax[it>en]]]: hi
* [15:12] Drew Dwi: maybe just disable scripts @ the estate?
* [15:12] Ovaltine Constantine: Since I have no way of seeing my script time
* [15:12] Indira Matova: ciao
* [15:12] Oskar Linden: ur taking up a combind 50ms of script time
* [15:12] Identity Euler: o,O
* [15:12] Ovaltine Constantine: Me personally? Or everyone here?
* [15:12] Oskar Linden: u
* [15:12] Drew Dwi: taking off Gaga Rings 2 might be a good start >.<
* [15:12] Indira Matova: xD
* [15:13] Oskar Linden: ansi comes in second with about 35 total
* [15:13] ANSI Soderstrom: how to move ?
* [15:13] Latif Khalifa: it's not scripts
* [15:13] Zwagoth Klaar: I'd just disable scripts for now, don't think anyone is using them too extensively.
* [15:13] Oskar Linden: good call
* [15:14] Oskar Linden: scripts disabled
* [15:14] Indira Matova: foxy:)
* [15:14] Drew Dwi: its agent time which is killing us, but I don't know exactly what goes into that outside of agent movement updates
* [15:15] Ratchet Xevion: Oskar, you killed Zwag and my eyes. lol
* [15:15] Oskar Linden: probably my eyes too
* [15:15] Geo Meek: wth
* [15:15] Xugu Madison: Can we make a start on this agenda?
* [15:15] Oskar Linden: yup
* [15:15] Identity Euler: 
* [15:15] Oskar Linden: ok so we have completed the pilot roll this week
* [15:16] Oskar Linden: Lil rolled out 1.36.0 to the central and the pilot regions
* [15:16] Latif Khalifa: 1) introduce concept of instrumentation so lindens can actually find out what is going on on a loaded sim
* [15:16] Oskar Linden: tentatively things look good
* [15:16] Drew Dwi: -lindens +estate managers
* [15:17] Zwagoth Klaar: both would be nice :>
* [15:17] Oskar Linden: we have the full roll of 1.36.0 out next week
* [15:17] Ovaltine Constantine: Yay, Ive been waiting forever for a way to get the parcel uuid
* [15:17] Oskar Linden: first half on tuesday
* [15:17] MartinRJ Fayray: Me too, Ovaltine.
* [15:17] Oskar Linden: second half on wednesday
* [15:17] Zwagoth Klaar: Sounds good to me.
* [15:18] Oskar Linden: anything you notice that is affecting 1.36 differently thatn 1.34 on the main grid should be filed in jira and brought to my attention
* [15:18] Zwagoth Klaar: hides his face.
* [15:18] Oskar Linden: anything super critical we can try and fix before the full roll
* [15:18] Oskar Linden: did you file that bug zwag?
* [15:18] Skills Hak: whoa full house
* [15:19] Zwagoth Klaar: Working on it right now. Although its still somewhat speculative. Although it seems to hold true for three regions I tested it on.
* [15:19] Eddi Decosta: wb Drew 
* [15:19] Oskar Linden: big crowd
* [15:19] Oskar Linden: that's awesome
* [15:19] Oskar Linden: so yeah. that's that for 1.36
* [15:19] Psi Merlin: 1.36 looks OK so far on my island.
* [15:20] Oskar Linden: we've got some tweaks planned for 1.36.1 and 1.36.2 already
* [15:20] Oskar Linden: 1.36.1 will contain some login server tweaks for search and welcom island load balancing
* [15:20] Oskar Linden: nothing too fancy
* [15:20] Oskar Linden: 1.36.2 has a few critical bug fixes
* [15:20] Oskar Linden: some for the linden homes project
* [15:20] Oskar Linden: and a few of a sensitive nature I cannot discuss
* [15:21] Oskar Linden: raises one eyebrow suggestively
* [15:21] Zwagoth Klaar: bites lip again.
* [15:21] Nigma Sterling: O.o
* [15:21] ANSI Soderstrom: like me ?
* [15:21] Latif Khalifa: linden sekrit stuff gets old, fast :P
* [15:21] Drew Dwi: what fixes are going in that you can talk about? >.>
* [15:21] Oskar Linden: I know latif
* [15:21] Oskar Linden: but you love me anyway
* [15:22] Geo Meek: any info?
* [15:22] Latif Khalifa: <3 LL :D
* [15:22] Oskar Linden: it's not the smartest thing to be like hey so here's this exploit, and we're rolling a fix for it in 3 weeks
* [15:22] Oskar Linden: then 3 weeks of either crashes or copybot stuff
* [15:22] Lares Carter: i was reading somewhere that SVC-3895 would be fixed with 1.36.1, but i guess thats not the case?
* [15:22] Oskar Linden: so cut me some slack
* [15:23] Geo Meek: why?
* [15:23] Oskar Linden: 1.36.1 is a super tiny interim hot fix
* [15:23] ATechwolf Foxclaw: Any way to find out the exploits after the fix goes gidewide. I'me thinking of regression testing on beta sims.
* [15:23] Oskar Linden: and is going out when it was scheduled to
* [15:23] Oskar Linden: 1.36.2 is the new 1.36.1
* [15:23] Latif Khalifa: i hear 1.38 is the new black xD
* [15:24] Oskar Linden: pretty sure the mono fix is in there lares
* [15:24] Skills Hak: thats what in 2 months?
* [15:24] Lares Carter: thanks oskar :)
* [15:24] Latif Khalifa: 6 weeks usually
* [15:24] Oskar Linden: 1.36.2 is planned for delivery march 2-3
* [15:25] Skills Hak: cool
* [15:25] Oskar Linden: 1.38 is end of march
* [15:25] sensory Hax: figo
* [15:25] Oskar Linden: we are working hard to get all the 1.38 stuff through qa this week and next
* [15:26] Xugu Madison: which means we'll see it on beta grid soon?
* [15:26] Latif Khalifa: ufortuatelly babbage said 1.38 contains only partal fix for freezes on rez
* [15:26] Oskar Linden: 1.38 will be here by the 22nd of Feb at the earliest
* [15:26] Latif Khalifa: the full fix is still being worked on
* [15:26] Latif Khalifa: will be interesting to test rez times
* [15:26] Oskar Linden: there is a lot of script related work going into 1.38
* [15:27] Skills Hak: so the 1.37 stuff will be rolled out with 1.38 then guess ?
* [15:27] Oskar Linden: I'm sure you'll all be eager to take a crack at it
* [15:27] Latif Khalifa: yup
* [15:27] Skills Hak: wohooo
* [15:27] Psi Merlin: Indeed.
* [15:27] Oskar Linden: 1.37 is the trunk revision number for what will be 1.38 in release
* [15:27] Latif Khalifa: llSetPrimitiveParamsFast ftw
* [15:27] Indira Matova: o_O
* [15:27] Zwagoth Klaar: \o/ llSPPF
* [15:28] Oskar Linden: so that's my next few weeks in a nutshell
* [15:28] Oskar Linden: hopefully I'll get over this cold soon
* [15:28] Latif Khalifa: Oskar, I know I'm always harping on the release notes, but for 1.38 will be extra important to documment all the scripting changes :D
* [15:28] Oskar Linden: we release all the notes we can
* [15:29] Oskar Linden: if the notes are not sufficient let me know and I'll see what more info I can get from babbage and crew ok?
* [15:29] Latif Khalifa: ok :) I'll torture him as well :D
* [15:30] Oskar Linden: so tonight we have some updates going on with the util server on aditi
* [15:30] Oskar Linden: we're rotating in a new machine
* [15:30] Oskar Linden: around 8PM
* [15:30] Oskar Linden: so logins may be affected, and if you are logged in I can't be certain what the experience will be like
* [15:30] Oskar Linden: so, go grab asome coffee and read a book :-)
* [15:31] Skills Hak: logins are for quitters
* [15:31] Oskar Linden: I will be maybe at a tequila bar
* [15:31] Latif Khalifa: lol
* [15:31] Laurent Bolissima: LOL
* [15:31] Oskar Linden: perhaps with womens
* [15:31] Oskar Linden: or not
* [15:31] Oskar Linden: who knows
* [15:31] Latif Khalifa: what about havok7 test
* [15:31] Oskar Linden: havok 7 testing is still ramping up
* [15:31] Oskar Linden: it is scheduled for 1.40
* [15:31] Zwagoth Klaar: That early? Woo!
* [15:31] Oskar Linden: yeah
* [15:32] Indira Matova: april?
* [15:32] Oskar Linden: falcon gave a nice little talk last week
* [15:32] Oskar Linden: you can read the transcripts
* [15:32] Skills Hak: makes note: need to go to falcon linden OH
* [15:32] Zwagoth Klaar: Considering the havok 4 took a few years, the transition to havok 7 is almost instant. :>
* [15:32] Oskar Linden: maybe early May
* [15:32] Oskar Linden: Falcon is from Havok
* [15:32] Indira Matova: cool:)
* [15:32] Oskar Linden: so he knows what he is doing
* [15:32] Oskar Linden: he mostly has performance enhancements
* [15:32] Oskar Linden: at this time for ground based vehicles
* [15:33] Zwagoth Klaar: Anything about joints? Or is that all just speculation at this point?
* [15:33] Oskar Linden: and some abilities for better integration with sculpts
* [15:33] Oskar Linden: well he told us last week he would love to have joints brought back
* [15:34] Oskar Linden: but didn't know how feasible it was
* [15:34] Oskar Linden: even though he pokes at it when he can
* [15:34] Oskar Linden: I will be messaging this group when we have regions here for early havok testing
* [15:35] Oskar Linden: hopefully in just a few weeks
* [15:35] Oskar Linden: Milo is leading the QA efforts internally
* [15:35] Oskar Linden: he was here all through the original H4 efforts
* [15:36] Oskar Linden: so zwag
* [15:36] Latif Khalifa: col
* [15:36] Oskar Linden: you wanna tell me about your bug
* [15:36] Latif Khalifa: cool stuff
* [15:36] Oskar Linden: or tell us
* [15:36] Latif Khalifa: physics chages i mean
* [15:36] Zwagoth Klaar: Tori and sphers now appear to take about 8 times as much physics time and don't settle the same way, if at all.
* [15:36] Oskar Linden: on 1.36 right?
* [15:36] Oskar Linden: has anyone else noticed that?
* [15:36] Zwagoth Klaar: dropping a single torus on the ground can take up 11ms of physics time in my testing.
* [15:37] Latif Khalifa: i haven't played with physical tori
* [15:37] Oskar Linden: did you file a jira yet zwag?
* [15:38] Zwagoth Klaar: I'm writing it.
* [15:38] Oskar Linden: file a shell, with your notes, and I'll have the rest of the group look at it and update notes
* [15:38] Oskar Linden: mark it as affecting server 1.36
* [15:38] Oskar Linden: so in the any other business section there was a quwstion about rolling deploys
* [15:39] Oskar Linden: whose was that?
* [15:39] Drew Dwi: <--
* [15:39] Zwagoth Klaar: 
* [15:39] Oskar Linden: yay
* [15:39] Lil Linden: HALLO THAR!
* [15:39] Oskar Linden: I asked lil to pop in drew
* [15:39] Oskar Linden: there he is
* [15:39] Skills Hak: []D [
* [15:39] Oskar Linden: Drew. please ask your question
* [15:40] Lil Linden: things is rezzin' slow for me
* [15:40] Lil Linden: IW and IRL today :)
* [15:40] Lil Linden: so speak slowly :)
* [15:40] Oskar Linden: Ahern is taking a pounding lil
* [15:40] Oskar Linden: so I disabled scripts
* [15:40] Lil Linden: aww.. but my animated hair!
* [15:40] Drew Dwi: the question is basicly based on an article I read [(http://torrentfreak.com/twitter-uses-bittorrent-for-server-deployment-100210)]
* [15:40] Oskar Linden: Lil is the hardest working Linden btw
* [15:41] Lil Linden: Drew - can you summarize?
* [15:41] Lil Linden: It would be boring to watch me read an article :)
* [15:41] Drew Dwi: short version is, instead of sending the entire file to each server one by one, in a 1:1 transfer, using the bitorrent server to collectively leverage the protocol and distribute updates faster
* [15:41] Lil Linden: That's a brilliant idea :)
* [15:41] Lil Linden: We already do it :)
* [15:41] Drew Dwi: sooo my question was, how is it done no?
* [15:41] Drew Dwi: now*
* [15:42] Lil Linden: Here's the deploy process in a nutshell
* [15:42] Lil Linden: 1. Oskar and I go into the kitchen, usually late at night, and bake up some new fangled server code. He makes sure it tastes good, and I just poke fun at him
* [15:42] Lil Linden: 2. Deploy day rolls around
* [15:43] Oskar Linden: that's true actually
* [15:43] Latif Khalifa: lol
* [15:43] Lil Linden: 3. I compile the server code and create a tarball of the binaries
* [15:43] Lil Linden: 4. I then put the tarball on the Asset System, so that it's globally available
* [15:43] Lil Linden: (to our systems)
* [15:44] Lil Linden: 5. For each colo, we start a bittorret tracker and have the simulator nodes start sucking down the tarball
* [15:44] Lil Linden: that usually takes < 1 hour
* [15:44] Lil Linden: 6. I send out a command to have each sim unpack the tarball and get it ready for prime time
* [15:44] Lil Linden: that's another hour or so
* [15:44] Lil Linden: per colo
* [15:45] Lil Linden: 7. When it's time for the rolling restart, we put the grid into what's called "Startup Mode"
* [15:45] Lil Linden: that means that when a sim goes down, the system won't try to bring the region up on another host
* [15:45] Oskar Linden: I stand around waiting for things to break and offer lil encouragements
* [15:45] Lil Linden: this way regions only get restarted once
* [15:46] Lil Linden: 8. While in startup mode, we take 200 sims at a time, shut down the simulator, install the new binaries, and then start it back up again
* [15:46] Lil Linden: repeat until all ~6000 sims are done
* [15:46] Lil Linden: and voila! A new version is born : )
* [15:46] Latif Khalifa: you mean hosts?
* [15:46] Lil Linden: sims = hosts for us
* [15:46] Drew Dwi: [15:45:33
* [15:47] Latif Khalifa: Lil, what do you call 4 things running on a single host?
* [15:47] Drew Dwi: that means it will try to bring it back up on that host? or it waits untill the rolling restart gets to it?
* [15:47] Lil Linden: well, ok, some jargon:
* [15:47] Latif Khalifa: (or 16 for opespace)
* [15:47] Lil Linden: "Sim" usually means "sim host", or the hardware
* [15:47] Zwagoth Klaar: believe they call them regions, the machine is a simhost
* [15:48] Lil Linden: "Simulator" is the binary that runs on the sim hosts.
* [15:48] Lil Linden: "regions" run in the simulator
* [15:48] Latif Khalifa: the other lindens usually equate sim and region
* [15:48] Lil Linden: that's a misnomer
* [15:48] Lil Linden: make fun of those lindens
* [15:48] Latif Khalifa: i'll tell andrew ;)
* [15:48] Lil Linden: Uh oh :)
* [15:48] Lil Linden: But I think he'd agree with me
* [15:49] Drew Dwi: I assume that startup mode is invisible to us?
* [15:49] Lil Linden: For the most part yes
* [15:49] Latif Khalifa: startup mode just makes regions sticky to sim hosts
* [15:49] Lil Linden: Do you want to hear more about startup mode?
* [15:49] Lil Linden: aka .... THE REGION CONDUCTOR? :)
* [15:49] Jonathan Yap: Yes
* [15:49] Latif Khalifa: regular restart can toss them anywhere
* [15:49] Lil Linden: queue dramatic music
* [15:49] Lil Linden: yeah ok, so!
* [15:50] Lil Linden: We have, on a "central server", a piece of software called the Region Conductor
* [15:50] Lil Linden: (central servers = everything besides simhosts)
* [15:50] Lil Linden: The region conductor has a simple job:
* [15:50] Oskar Linden: lil just got out of andrews server architecture class so this si fresh on his mind
* [15:50] Lil Linden: 1. look for regions that should be running, but are down
* [15:50] Lil Linden: 2. start those regions on a sim host that has room for it
* [15:51] Latif Khalifa: does it try to keep them in the same colo?
* [15:51] Lil Linden: when the grid is in normal mode (aka "Up Mode") it starts the region on a random simhost
* [15:51] Lil Linden: latif - yes, we usually keep the regions in the same colo
* [15:51] Drew Dwi: how many colo's currently? still just 2?
* [15:51] Lil Linden: we have 4 actually
* [15:52] Lil Linden: temporarily
* [15:52] Lil Linden: San Fran, Phoenix, Dallas, and a fancy new one
* [15:52] Latif Khalifa: they added that "green" one last year that started off losing power once a month or so;)
* [15:52] Drew Dwi: east cost?
* [15:52] Lil Linden: yeah when the wind stops everythign shuts down :)
* [15:52] Drew Dwi: coast even
* [15:52] Lil Linden: Yes
* [15:52] Latif Khalifa: it's so green it uses no power from time to time :P
* [15:52] Drew Dwi: lol
* [15:52] Lil Linden: :)
* [15:52] Lil Linden: So anyway
* [15:53] Oskar Linden: thankfully we just moved it closer to the SF office that is full of hot air
* [15:53] Zwagoth Klaar: Is there anything special that happens to simulator processes that become stuck or enter an infinite loop of some kind? Are they killed off automatically or do they get seen to by hand?
* [15:53] Lil Linden: if we did sa rolling restart with the grid in Up Mode, then when your region shut down, it would move to another sim on the old version
* [15:53] Lil Linden: then your region would get restarted again when the rolling restart got to it
* [15:53] Lil Linden: and again.. until eventually you hit a new version sim
* [15:53] Lil Linden: that would be annoying
* [15:54] Latif Khalifa: yeah making them stick to hosts makes all the sense
* [15:54] Lil Linden: Zwagoth: yes
* [15:54] Lil Linden: we watch for sims that are unresponsive and kill them off.. There is another process that looks for sims that are not running and starts them
* [15:54] Latif Khalifa: will be fun when the planned european colo goes online :)
* [15:54] Lil Linden: that way things don't get stuck
* [15:54] Lil Linden: Latif++!
* [15:54] Lil Linden: It's in Amsterdam!
* [15:55] Lil Linden: I wanna go
* [15:55] Oskar Linden: metoo
* [15:55] Latif Khalifa: i'm closer!
* [15:55] Lil Linden: I'll stay at your place :)
* [15:55] Oskar Linden: but you don't have keys
* [15:55] Lil Linden: true :(
* [15:55] Oskar Linden: alright party at latif's
* [15:55] Lil Linden: haha
* [15:55] Zwagoth Klaar: I'm guessing unresponsive is about 20 minutes? That seems to be how long a sim stays down if it enters a crash loop.
* [15:55] Eddi Decosta: party?
* [15:55] Lil Linden: zwagoth: unsure of the timing
* [15:55] Oskar Linden: we can talk about all the comments he's tossed out in AW Groupies chat
* [15:55] Latif Khalifa: i could turn to the dark side of the force and apply for an ops position at LL :P
* [15:55] Lares Carter: i thought the amsterdam office is just public relationships?
* [15:55] Lil Linden: haha
* [15:56] Lil Linden: Lares: it's still all up in the air.. we'll see what actually happens
* [15:56] Eddi Decosta: well, i need to resend my resume? lol
* [15:56] Lil Linden: We're always looking for smart people :)
* [15:57] Latif Khalifa: actually i already do something similar. high performance clustering for genome sequencing apps ;)
* [15:57] Lil Linden: So did anyone have any other questions on the restart process? Drew did that answer your question?
* [15:57] Lil Linden: awesome latif!
* [15:57] Drew Dwi: yes, very interesting :)
* [15:57] Latif Khalifa: (much smaller scale than linden grid though)
* [15:57] Lil Linden: Can you get me one of those cool pictures of my DNA? :)
* [15:57] Eddi Decosta: question out subject, why my l$ said Loading...
* [15:57] Lil Linden: haha
* [15:57] Lil Linden: Oh THAT
* [15:57] Latif Khalifa: hehehe
* [15:57] Lil Linden: So the $L thing is unique to aditi
* [15:58] Jonathan Yap: Lil, all this would make a great wiki entry/article
* [15:58] Lil Linden: Jon: feel free to write it up :)
* [15:58] Lil Linden: I would love to, but I'm swwwaaamped
* [15:58] Oskar Linden: I will be posting the transcript to the wiki
* [15:58] EriX Oh: Datacenter: why not to choose Luxembourg, to put it... since we have here new Fiberlines (Terralink). e.t.c. and the VAT is more cheaper... :-)
* [15:58] Lil Linden: So the $L issue is because SL Grids are COMPLICATED
* [15:59] Lil Linden: Agni has over 100 "central servers", machines that aren't sim nodes
* [15:59] Lil Linden: to keep things running
* [15:59] Latif Khalifa: i've seem problems with it on agni too, meant to bother Which Linden today but missed his OH today :(
* [15:59] ATechwolf Foxclaw: returns "My $L is stuck at "Loading" too."
* [15:59] Lil Linden: We overload machines on Aditi (give one machine two jobs or more) because it's not practical to duplicate all that
* [16:00] Lares Carter: My L$ get stuck everytime i login to an 1.34. sim. Works when fine when i start in a 1.36 sim.
* [16:00] Lil Linden: One of the changes in 1.36 assumes that there is a central server to route some new $L requests
* [16:00] Lil Linden: Aditi does not have that server, so those requests get dropped
* [16:00] Latif Khalifa: Enus was talking about using Amazon EC2 to launch 80k bots to simulate production concurrency on a test grid
* [16:01] Zwagoth Klaar: Sounds hideously expensive.
* [16:01] Lil Linden: Once we upgrade the Aditi sims to 1.36 as well, though, that problem should go waway
* [16:01] Lil Linden: *away
* [16:01] Latif Khalifa: Zwagoth, ec2 is quite cheap for short test runs like that
* [16:01] Lil Linden: well...
* [16:01] Lil Linden: in aggregate it adds up :)
* [16:02] Lil Linden: wouldn't want to pay the bill
* [16:02] Latif Khalifa: you can start 200 clients from a single instance ;)
* [16:02] Lil Linden: hehe.. camper bot army!
* [16:03] Oskar Linden: maestro got a $0.02 bill from amazon the other day for a test he ran
* [16:03] Lil Linden: .. or a really boring dance party
* [16:03] Zwagoth Klaar: volunteers Latif to pay for 500 instances.
* [16:03] Oskar Linden: well we are over the hour
* [16:03] Oskar Linden: this has been a good meeting
* [16:03] Oskar Linden: thank you much lil for gracing us with your pimpness
* [16:03] Lil Linden: bows
* [16:04] Drew Dwi: thanks for info :)
* [16:04] Drew Dwi: I toyed with pyogp, but don't have that many accounts :P
* [16:04] Latif Khalifa: we have a small competition going if you could start more using libomv or pyogp
* [16:04] Latif Khalifa: Lil, actually it was Enus who tested it
* [16:04] Drew Dwi: pyogp for sure, libomv is a memory hog
* [16:05] Latif Khalifa: not true at all
* [16:05] Lil Linden: nice talking to you guys! thanks for the good questions :)
* [16:05] Latif Khalifa: Enus was able to login 400 of his tester bots using libomv from a single windows box
* [16:05] Drew Dwi: at least last time I played with it
* [16:05] Xugu Madison: thanks for coming Oskar & Lil!
* [16:05] Latif Khalifa: Lil, thaks fr informative meeting :)
* [16:05] Lil Linden: needs to go work on baking that new version
* [16:05] Latif Khalifa: Oskar. thanks for your time :)
* [16:05] Drew Dwi: thanks!
* [16:05] Kyra Vixen: Have a good day Oskar, Lil.
* [16:05] Eddi Decosta: i need to log off with Yann, the second client have a big lag issue :p
* [16:06] Latif Khalifa: Drew, just ask Enus :)
* [16:06] Zwagoth Klaar: Make sure to turn scripts back on in the region before you poof Oscar :>
* [16:06] Oskar Linden: thanks for coming everyone
* [16:06] Zwagoth Klaar: And thanks for having the OH
* [16:06] Indira Matova: thank you
* [16:06] Oskar Linden: i will zwag
* [16:06] Eddi Decosta: thanks to you Oskar 
* [16:06] Latif Khalifa: yeah, one of the best OH
* [16:06] Psi Merlin: /Thanks Oskar,Lil
* [16:06] Latif Khalifa: should put it in my calendar
* [16:06] Oskar Linden: anyone that wants to test out zwags bug it is here: 
* [16:07] Jonathan Yap: The conductor program needs to give Morris a big poke
* [16:07] Eddi Decosta: and lil, sorry i never see you ..
* [16:07] Oskar Linden: we're looking for new to 1.36 poor perfromance for physical objects
* [16:07] Zwagoth Klaar: will have to import the relevant object.
* [16:07] Eddi Decosta: yeah, i need to look 
* [16:08] Eddi Decosta: you dont have a objection if i use a modified popgun on oatmeal? LOL
* [16:08] Oskar Linden: I never have eddi
* [16:08] Eddi Decosta: oh right