<transcript><audio_222> de ir aprendiendo y adaptándonos también a esta nueva situación que nos toca vivir y yo creo que ciertamente sí, pudiendo ofrecer claramente a los casos que sí lo requieren y que están graves un tratamiento</transcript><transcript><audio_223> menos inmediato y que no se aíslen en su entorno sin acudir a los dispositivos que existen. Muy bien, muchas gracias. Hay una pregunta en el</transcript><transcript><audio_224> a si hay un nuevo confinamiento total, qué acciones, con la experiencia que ya tenemos de la primera ola, qué acciones tendríamos que tener en cuenta tanto para fracasar</transcript><transcript><audio_225> y no es que lo sepamos, que tengamos una varita mágica, pero pensando en la situación anterior, normalizar rápidamente, sí, en primer lugar, un tratamiento virtual. No que en ese momento lo iniciemos,</transcript><transcript><audio_226> el mantener de forma combinada uno y otro y que por tanto en un momento determinado que hay un empeoramiento en cuanto a las condiciones de aislamiento o de confinamiento que podamos acceder</transcript><transcript><audio_227> a esa situación, a un tratamiento si es de grupo, si es individual, etcétera, etcétera. Que tal vez a nivel de, y lo estamos haciendo, de comunicación entre las distintas unidades, que no interrumpa.</transcript> well, maybe we also have to learn and adapt to this new situation.  that we have to live and I believe that we can certainly offer clearly to the cases that do require us and that are serious a more or less immediate treatment and that they do not isolate themselves in their environment without resorting to the devices that exist well thank you very much there is a  Ask in the chat in relation to whether there is a new total confinement, what actions with the experience we already have from the first wave, what actions would we take into account for both Well, on our part, at least it is not that we know that we have a magic wand but thinking about the previous situation, quickly normalize whether in the first place a virtual treatment or not that at that moment we start it not like everything we are doing is maintaining both in a combined way and therefore at a certain moment there is a  worsening in terms of the conditions of isolation or confinement that we can access in the least traumatic way possible to that situation and to a treatment if it is a group if it is individual etcetera etcetera that perhaps at the level of what we are doing in terms of communication between the different units  that we do not increase


<transcript><audio_228> la comunicación, que no se pierdan esos pacientes en la transición entre infanto, juvenil, tal vez, y adultos, y eso yo creo que ahí estamos trabajando mucho, y luego también, lo comentaba previamente,</transcript> the dialogue as we are doing, the communication that those patients are not lost in the transition between infant-adolescent and perhaps already adults and that I believe that we are working there a lot and then Sarah also commented on it to

Creative Commons Attribution license (reuse allowed)


<transcript><audio_229> también, es decir, esos casos que tal vez son más reticentes, pues porque a lo mejor no presentan sintomatología tan clara en acudir a primaria o en acudir a</transcript><transcript><audio_230> Pues que lo hagan, que lo hagan y que encontremos vías, bien sean telefónicas o de otra forma, que agilicen el paso a una unidad especial en caso de que sea necesario.</transcript><transcript><audio_231> porque el pasado miércoles con un grupo de terapia que hacemos los miércoles con los pacientes, quisimos analizar un poco el tema del confinamiento y de la mitad de los pacientes éramos un grupo de</transcript><transcript><audio_232> recaído durante el confinamiento y la mitad eran nuevos casos. Entonces, después de analizar un poco lo que hemos hablado aquí, les preguntamos, bueno, y si ahora no tuviéramos que dar un mensaje a</transcript><transcript><audio_233> sin trastorno alimentario, ¿no? Después de vuestra experiencia, ¿qué cosas haríais diferente? Un poco también contestando esta pregunta, pero desde la versión de los propios pacientes, ¿no? Pues ellos hablaban de mantenerse</transcript><transcript><audio_234> como aprendizaje. Hacer cosas que te gusten, muy importante. Hablaban de la supervisión en las comidas. Muchas veces decíamos, pero bueno, si los padres estaban en casa,</transcript> Montse it is also  say those cases that perhaps are more reticent because perhaps they do not present such clear symptoms in going to primary school or in going to more community facilities, well, let them do it, let them do it and let us find and diaz, whether by telephone or in another way that  expedite the transfer to a special unit if necessary, thank you very much, you will allow me because last Wednesday with a therapy group that we do on Wednesdays with the patients we wanted to analyze a little the issue of confinement and half of the patients, we were a group of ten, half of the patients had relapsed during confinement and half were new cases. After analyzing a little what we have talked about here, we asked well, what if now we didn't have to give a message no to the people who  is in eating disorder not after your experience what things would you do differently a little also answering this question but from the version not of the patients themselves because they talked about keeping the mind busy not as learning not doing things that you like not very important they talked  about supervision at meals, not many times we said, testing if the parents were at home,


<transcript><audio_235> Estábamos acompañados. El hecho de establecer rutinas. Daban valor a las videollamadas de amigos. Ellos mismos nos decían</transcript><transcript><audio_236> Es que muchas veces las respuestas las tenemos ahí cerca con los pacientes. Seguir el tratamiento telemático. Mira, Fernando, como comentabas tú, seguir el tratamiento y después salió...</transcript><transcript><audio_237> No sabemos si por los beneficios de tener un animal o por poder salir en caso de confinamiento. Bueno, hay muchas preguntas, muchas en relación a las dificultades de acceso. Las que no podamos responder os damos</transcript><transcript><audio_238> que es hola arroba som360.org y trataremos de responderlas vía online. A mí me gustaría para concluir decir</transcript> was no supervision, we were accompanied, not the fact of establishing routines gave value to the video calls where friends with themselves told us to control the networks.  Social is that many times we have the answers nearby, not with the patients, following the Telemann treatment of eating Fernando, as you mentioned, not following the treatment and then it came out like this, like an anecdote with a dog at home, we do not know if it is because of the benefits of having an animal.  or or to be able to go out in case of confinement, well, there are many questions in relation to access difficulties that we cannot answer, we will give you an email that is or the at sign is 360 point heretic and we will try to answer them via online to me To conclude, I would like to conclude,


<transcript><audio_239> Una pequeña ronda a los tres ponentes y con lo que consideréis para finalizar este webinar algún mensaje de cara a las familias, de cara a las personas afectadas.</transcript><transcript><audio_240> ¿Qué pasa con los pacientes con discapacidad?</transcript><transcript><audio_241> es que a pesar de que estamos en plena pandemia, a pesar de que el acceso a los tratamientos es dificultoso o más dificultoso que habitualmente, se puede acceder. Y de hecho, en determinados casos no se puede esperar mucho.</transcript><transcript><audio_242> que se pueda acabar accediendo. Y también, pues, insistir en los mensajes que enviamos siempre, ¿no? En el hecho de que, bueno, pues, hay que informarse sobre que es un trastorno de conducta alimentaria porque la información es poder en este aspecto, ¿no? Y muchas veces la familia ha</transcript><transcript><audio_243> Muchas veces la reacción natural que llevan a cabo, sin conocer que es un trastorno de conducta alimentaria y cuáles son las pautas más recomendables, a menudo no son las más útiles. Aunque sean naturales, no son las más útiles. Por tanto, es muy importante que escuchen a las personas que se deshacen</transcript> yes, well, we are going to do a small round with the three speakers and with what you consider to end this webinar, a message for the families, for the affected people and also colleagues who are fighting or not alongside the  family and the patient to be able to get out of this disease, well because, above all, to the families, one of the main messages we would send them is that even though we are in the midst of a pandemic, to think that the  Access to treatments is difficult, more difficult than it usually is to access, that is, you have to go, in fact, in certain cases, you cannot wait for them to ask for help, they insist that it can end up being accessed, and also because they insist on the messages that we always send in  the fact that well, you have to find out what an eating disorder is because information is power in this aspect, there are often times when the family is faced with this situation and we were often deceived by the natural reaction that they carry out, not without knowing what it is.  a complementary person and what are the most recommended guidelines are often not the most useful, even if they are natural, they are not the most useful, therefore it is very important that you listen to the versions


<transcript><audio_244> un poquito en cómo atender a la persona en casa porque aunque cueste es dificultoso es todo un proceso igual que tiene que pasar con la persona afectada pero se aprende y vale mucho la pena porque una familia que</transcript><transcript><audio_245> psicológico para manejar mejor esta o apoyo emocional para manejar esta situación favorece un mejor pronóstico en la persona afectada que hagan mucho equipo como decía antes con los profesionales con la persona afectada también que sean competentes</transcript> that you allow yourself to be advised to train a little, not in how  Caring for the person at home because although it is difficult, it is a whole process, the same as what the affected person has to go through, but they do learn and it is very worth it because a family that receives and receives psychological support to better manage this emotional support to manage this  situation favors a better prognosis in the affected person, that they make a lot of teamwork, as I said before, with the professionals with the affected person, also that they are understanding,


<transcript><audio_246> Que la persona está secuestrada, por así decirlo, por una enfermedad mental que es grave y que paciencia que va para largo y que todo se supera, ¿no? Que recuerden que es un trastorno que con tratamiento y con apoyo y con paciencia pues existe la posibilidad.</transcript><transcript><audio_247> de recuperación total y de hecho en la asociación tenemos personas voluntarias que son personas que han superado con éxito un trastorno de conducta alimentaria, algunas ya hace muchos años y que actualmente llevan una vida completamente plena y normalizada. Y bien,</transcript> that they understand that it is a disorder that the person is kidnapped, so to speak, with a mental illness that is serious and that patience  that it goes for a long time and that everything is overcome, do not remember that it is a treatment, it is a disorder that with treatment and with support and with patience because there is the possibility of recovery and total recovery and in fact the association has volunteer people who are people who  They have successfully overcome an eating disorder some and many years ago and they currently lead a completely full and normalized life.


<transcript><audio_248> Gracias. Por mi parte igual decir, me uno evidentemente a lo que comenta Sara, básicamente para pacientes, para familiares, para compañeros, estamos aquí, lo haremos mejor o peor, lo estamos</transcript><transcript><audio_249> vamos a estar a vuestro lado y queremos ayudaros, confiad en nosotros, claramente sabemos ahora sí lo que es manejar la incertidumbre y las dificultades que teníais, ahora las entendemos bien y no es tan fácil, pero</transcript> I will be here, thank you for my part. I will obviously agree with what Sara comments, basically for patients, for relatives, for colleagues, we are here.  We will do it better or worse. We are trying. We are going to be by your side and we want to help each other. Trust us. We clearly know now if what it is like to handle the uncertainty and the difficulties that you had.


<transcript><audio_250> Con lo cual, confiad y aquí estamos. Pues, bueno, yo por supuesto me uno a lo que han dicho mis compañeros y</transcript><transcript><audio_251> diría yo, para profesionales pues un poquito ver que hemos sabido salvar la situación, hemos sido rápidos yo creo que los programas de teleterapia con</transcript><transcript><audio_252> combinada lo presencial y la teleterapia tiene su efecto, creemos que ha sido efectivo para mantener la continuidad, animo a los profesionales a continuar con ellos.</transcript><transcript><audio_253> yo creo que se resume bien para los profesionales, ¿no? Me he sentido acompañado, no estaba tan solo cuando me llamabais y sabía que os ocupabais de mí. Esto, bueno, pues yo creo que esa es una buena forma de decirlo, lo que quiere</transcript><transcript><audio_254> Mensaje para las familias, pues lo ha dicho muy bien Sara, ¿no? Confíen, trabajen conjuntamente con nosotros, infórmense, hay documentos informativos, hay asociaciones.</transcript><transcript><audio_255> Y para los gestores, permitidme que también para los gestores, la salud mental es una ola y esto no es solo una frase y dentro de la salud mental los trastornos de la conducta alimentaria desaparecen.</transcript> trust and because of course I agree with what my colleagues have said and well, I would say about 33 messages for professionals, well, a little to see that we have known how to save the situation, we have been fast, I believe that the combined teletherapy programs on Friday  Using the presence of teletherapy in combination has its effect, we believe that it is effective in maintaining continuity, encouraging the professionals to continue with them and as one of our patients said in I think it sums up well for the professionals, I have felt supported.  I was not alone when you called me and I knew that you take care of me, this is good, well, I think that is a good way to say what I want to say, a message for the families, because Sara said it very well, do not trust, work together with us reports  and inform them if there are informative documents, there are associations, there are webinars, you are not alone, you are not alone, and for managers, let me also say that for managers, mental health is one and this is not just a phrase, and within mental health, behavioral disorders food,


<transcript><audio_256> en estas condiciones que genera la pandemia y por tanto hay que arbitrar, hay que estar atentos a proporcionar recursos para ellos también en estos momentos. Muchas gracias. Muy bien, pues muchas gracias Montse.</transcript><transcript><audio_257> Fernando, ha sido un placer poder compartir este webinar. Yo decir también que somos un equipo, profesionales, pacientes,</transcript><transcript><audio_258> estamos para ayudar y tenemos un recorrido también importante y tenemos que mantenernos y eso también va</transcript><transcript><audio_259> anunciarles que el próximo webinar es el 1 de diciembre a las 6 y sobre el impacto de la COVID-19 esta vez sobre las adicciones pues hasta aquí un saludo</transcript> I think they suffer especially in these conditions that generate the pandemic and therefore we have to arbitrate, we have to be attentive, it is not going to provide resources for them also at this time, thank you very much, very good, thank you very much mons, thank you very much, Ara and Fernando, it has been a  pleasure to and I also say that we are a professional team of patients and families and that we are here to help and we also have an important journey and we have to keep going well to meet the professionals and announce to them that the next  webinar is on December 1 at 6 and about the impact of cob and 19 this time on addictions so far a


<transcript><audio_260> Muchas gracias a los ponentes de nuevo y a los asistentes.</transcript> big greeting to all many thanks to the speakers again and to the attendees and


<transcript><audio_0>all right shabam I think you have a couple announcements you had something to share with the class yeah yeah Happy Diwali to everyone how was your Diwali it was good it was good but it's like uh shifted right so the Diwali in India just happened but then there's a US</transcript>
<image_1>The image is a slide from a presentation on multi-version concurrency control. The slide has a white background with black text. On the left side of the slide, there is a title that reads "Multi-Version Concurrency Control" and below it, there are two smaller images. The first image on the slide shows a man standing in front of a large screen, which appears to be a presentation slide. The second image shows the man standing on a stage, with a blue sky in the background. The text on the screen reads "MVCC is more than just a concurrency-control protocol. It completely affects how the DBMS manages transactions and the database."
<transcript><audio_2>Diwali that's happening the next weekend because everyone has to do it twice so yeah I mean that was nice and my girlfriend visited me all the weekend and she was really interested in finding who's sending flowers and Who coming to the lab so that was an interesting</transcript>
<image_3>The image is a slide from a presentation. It has a white background with black text. The title of the slide reads "Multi-Version Concurrency Control". Below the title, there is a subtitle that reads "MVCC is more than just a concurrency control protocol. It completely affects how the DBMS manages transactions and the database."

On the right side of the image, there are two images - one of a man sitting at a desk and the other of a computer monitor. The man is wearing a blue shirt and appears to be giving a presentation on the topic.
 all right shabam I think you have a couple announcements you had something to share with the class yeah yeah Happy Diwali to everyone how was your Diwali it was good it was good but it's like uh shifted right so the Diwali in India just happened but then there's a US Diwali that's happening the next weekend because everyone has to do it twice so yeah I mean that was nice and my girlfriend visited me all the weekend and she was really interested in finding who's sending flowers and Who coming to the lab so that was an interesting


<transcript><audio_4>conversation thanks to Andy and that's and I might be playing a couple of shows next weekend so I I'll post you guys on that as well and enjoy great wonderful thank you all right we have a bunch to</transcript><transcript><audio_5>cover today so we have a few things left over from the multiversion concurrency control part that we just started discussing last class and I'm going to spend about like 20 25 minutes on the mvcc and there's some portions of this</transcript> conversation thanks to Andy and that's and I might be playing a couple of shows next weekend so I I'll post you guys on that as well and enjoy great wonderful thank you all right we have a bunch to cover today so we have a few things left over from the multiversion concurrency control part that we just started discussing last class and I'm going to spend about like 20 25 minutes on the mvcc and there's some portions of this


<transcript><audio_6>that uh depending on how much we make through uh today uh might just at to stuff that you read on your own that won't be material for the exam but I'll tell you when we hit that point okay and then what we want to do for the bulk of</transcript><transcript><audio_7>today is to get started on the recovery component which is going to be split across uh today and the next lecture so if you remember we were doing this multiversion congruency control so if I go back a couple slides we were</transcript><transcript><audio_8>essentially doing things where as changes were happening to these objects we were creating these uh uh chains of of objects so let's go to slide seven where there you go so you saw how in this case a was being read and written a</transcript> that uh depending on how much we make through uh today uh might just at to stuff that you read on your own that won't be material for the exam but I'll tell you when we hit that point okay and then what we want to do for the bulk of today is to get started on the recovery component which is going to be split across uh today and the next lecture so if you remember we were doing this multiversion congruency control so if I go back a couple slides we were essentially doing things where as changes were happening to these objects we were creating these uh uh chains of of objects so let's go to slide seven where there you go so you saw how in this case a was being read and written a


<transcript><audio_9>bunch of times and what was happening in the database is we were keeping track of all those versions We had a begin and end time stamp that basically told us what is that value like a z is valid from 0 to one from 1 to two there's a</transcript><transcript><audio_10>different value and two onwards till sometime in the future uh there's a third value so that's what multiversion does it's going toate create these different versions and now we are the readers are going to come in and find the right versions to work with so</transcript><transcript><audio_11>multiversion is really really popular and in fact uh if you look at the systems that use it just about every modern system will use multiversion concurrency control okay now we'll go</transcript> bunch of times and what was happening in the database is we were keeping track of all those versions We had a begin and end time stamp that basically told us what is that value like a z is valid from 0 to one from 1 to two there's a different value and two onwards till sometime in the future uh there's a third value so that's what multiversion does it's going toate create these different versions and now we are the readers are going to come in and find the right versions to work with so multiversion is really really popular and in fact uh if you look at the systems that use it just about every modern system will use multiversion concurrency control okay now we'll go


<transcript><audio_12>and dig into the details of the other components that we need besides mvcc right we need a version management component as we talked about and with those versions you can decide which version a reader is allowed to see based upon their timestamp but we have a couple other things to discuss about the</transcript> and dig into the details of the other components that we need besides mvcc right we need a version management component as we talked about and with those versions you can decide which version a reader is allowed to see based upon their timestamp but we have a couple other things to discuss about the


<transcript><audio_13>mvcc design considerations so we're going to try and knck through most of this uh today and see how far we can get okay and we'll we are time bucketing about 25 minutes to cover these this component so we've talked about a whole</transcript><transcript><audio_14>bunch of congruency control protocols optimistic congruency control two- face locking so the natural question you be asking is how is this related to mvcc so the best way to think about this is that mvcc is a mechanism that tells</transcript><transcript><audio_15>you how to maintain multiple versions you still need a way to protect that mechanism with a concery control protocol multiversion basically creates a link list of different versions and at some point there's some</transcript><transcript><audio_16>writer that's potentially writing and creating a new version right we saw in in those examples so obviously you have to guard against things like two people trying to write and create new values like if you have a single link list only one person can add to that link list uh</transcript><transcript><audio_17>at a time right so only one person can be creating a new version at a time and so we're going to have a concery control mechanism that we need in addition to the mechanism of keeping these versions what this version stuff allows is that</transcript><transcript><audio_18>if I am a transaction that is reading something and my Tim stamp allows me to do that I can read an older version and the readers can get past the writers the writers don't block the readers if the readers have the right Tim stamp or they</transcript> mvcc design considerations so we're going to try and knck through most of this uh today and see how far we can get okay and we'll we are time bucketing about 25 minutes to cover these this component so we've talked about a whole bunch of congruency control protocols optimistic congruency control two- face locking so the natural question you be asking is how is this related to mvcc so the best way to think about this is that mvcc is a mechanism that tells you how to maintain multiple versions you still need a way to protect that mechanism with a concery control protocol multiversion basically creates a link list of different versions and at some point there's some writer that's potentially writing and creating a new version right we saw in in those examples so obviously you have to guard against things like two people trying to write and create new values like if you have a single link list only one person can add to that link list uh at a time right so only one person can be creating a new version at a time and so we're going to have a concery control mechanism that we need in addition to the mechanism of keeping these versions what this version stuff allows is that if I am a transaction that is reading something and my Tim stamp allows me to do that I can read an older version and the readers can get past the writers the writers don't block the readers if the readers have the right Tim stamp or they


<transcript><audio_19>have the appropriate serialization serialization order where they can pass through right in Two-Face locking if I grab the right lock there's only one version everyone has to wait for it a reader has to wait for it right and that's where we started to relax things</transcript><transcript><audio_20>with the intentional locking and things like that but still writers will block readers in the end case with two- face locking okay so the congruency control protocols you have you can say I can do a multiversion which is a mechanism to</transcript><transcript><audio_21>allow what can be read at what time you're keeping these different versions around can be combined with optimistic congruency control where you'll just run the three-phase protocol except in your private workspaces where you'll keep these new versions or two- face locking where you'll use locking for objects</transcript><transcript><audio_22>that are getting created like the writers not interfering with writers or if a reader needs to read that version that is being written because the time stamp say you need to be at a version that is being written they'll have to wait for that okay so locking can be</transcript><transcript><audio_23>used to do that uh you can also do something very simple we're not going to talk about that but time stamp ordering where you have some mechanism for picking a timestamp and then use that to determine the serial order effectively all that says you can come up with even simpler mechanisms where you say I grab</transcript> have the appropriate serialization serialization order where they can pass through right in Two-Face locking if I grab the right lock there's only one version everyone has to wait for it a reader has to wait for it right and that's where we started to relax things with the intentional locking and things like that but still writers will block readers in the end case with two- face locking okay so the congruency control protocols you have you can say I can do a multiversion which is a mechanism to allow what can be read at what time you're keeping these different versions around can be combined with optimistic congruency control where you'll just run the three-phase protocol except in your private workspaces where you'll keep these new versions or two- face locking where you'll use locking for objects that are getting created like the writers not interfering with writers or if a reader needs to read that version that is being written because the time stamp say you need to be at a version that is being written they'll have to wait for that okay so locking can be used to do that uh you can also do something very simple we're not going to talk about that but time stamp ordering where you have some mechanism for picking a timestamp and then use that to determine the serial order effectively all that says you can come up with even simpler mechanisms where you say I grab


<transcript><audio_24>a timestamp let's say at the beginning of the transaction and I only read versions as of my time stamp which is kind of what snapshot isolation does so you can get that type of protocol what we going to concern ourselves with is these other topics now right there are</transcript><transcript><audio_25>five things we need to talk about in addition to mvcc as a mechanism that creates this version the first one was concery control they're going to start knocking these other things down like next thing we're going to look at is how is the storage for this version</transcript><transcript><audio_26>maintained right it's a singly link list that we are maintaining what are the different ways of organizing that just as we looked at what are different ways of organizing records in pages and and uh structures like that so let's get to the version</transcript><transcript><audio_27>storage now we are creating these version chains and these version chains have these begin and end time stamps that are associated with it and there are different ways in which we could create these version chains as we'll see</transcript><transcript><audio_28>for all of the remainder topics that we have for mvcc for each of those mechanisms the each of these addition things that we need to consider for mvcc we'll see there are some number of options there'll be three four five options and then each one of them will</transcript> a timestamp let's say at the beginning of the transaction and I only read versions as of my time stamp which is kind of what snapshot isolation does so you can get that type of protocol what we going to concern ourselves with is these other topics now right there are five things we need to talk about in addition to mvcc as a mechanism that creates this version the first one was concery control they're going to start knocking these other things down like next thing we're going to look at is how is the storage for this version maintained right it's a singly link list that we are maintaining what are the different ways of organizing that just as we looked at what are different ways of organizing records in pages and and uh structures like that so let's get to the version storage now we are creating these version chains and these version chains have these begin and end time stamps that are associated with it and there are different ways in which we could create these version chains as we'll see for all of the remainder topics that we have for mvcc for each of those mechanisms the each of these addition things that we need to consider for mvcc we'll see there are some number of options there'll be three four five options and then each one of them will


<transcript><audio_29>have their pros and cons so the first question we are going to look at with version storage is how do we store these versions guess what there are three ways to do it both of all of them have their pros and cons and some historical context the first one is a pendon</transcript><transcript><audio_30>storage where I'm going to create the chains kind of like what we've been seeing in the diagram so far where in the same table where the record is you're going to create the version uh version</transcript><transcript><audio_31>way compon of those records time travel which is used by systems that didn't have mvcc but then realize that wow with mvcc you can get a lot more concurrency especially for the readers so they slapped on a mechanism to keep versions by saying I'll just create a second copy</transcript><transcript><audio_32>of the table there all my versions are maintained and we'll see that in a little bit the preferred way is Delta storage so if you're building an MVC system from scratch that's kind of what you will use so what's the simplest it's kind of what</transcript><transcript><audio_33>a diagrams have been so here I've got two records A and B and the entire records even though only one values being shown like uh 111 in the first one think of it as being the entire record that's what's getting</transcript><transcript><audio_34>stored in the table and as you create new versions you will create that essentially as a new record in that same table in that same physical table or the physical file that is keeping track of all the records in there because the new</transcript> have their pros and cons so the first question we are going to look at with version storage is how do we store these versions guess what there are three ways to do it both of all of them have their pros and cons and some historical context the first one is a pendon storage where I'm going to create the chains kind of like what we've been seeing in the diagram so far where in the same table where the record is you're going to create the version uh version way compon of those records time travel which is used by systems that didn't have mvcc but then realize that wow with mvcc you can get a lot more concurrency especially for the readers so they slapped on a mechanism to keep versions by saying I'll just create a second copy of the table there all my versions are maintained and we'll see that in a little bit the preferred way is Delta storage so if you're building an MVC system from scratch that's kind of what you will use so what's the simplest it's kind of what a diagrams have been so here I've got two records A and B and the entire records even though only one values being shown like uh 111 in the first one think of it as being the entire record that's what's getting stored in the table and as you create new versions you will create that essentially as a new record in that same table in that same physical table or the physical file that is keeping track of all the records in there because the new


<transcript><audio_35>record has the same schema all of them now have this additional what is shown as a pointer but that pointer is like that big and end timestamp and bunch of other information that's associated with it so you'll just create the new version and then you'll maintain the pointers internally right so you can think of it</transcript><transcript><audio_36>as your slotted page organization being used to create a file and in that file you have record IDs so a z is a record ID A1 is a different record ID but there's this extra field of pointer that chains them together and allows you to maintain your singly link list right</transcript><transcript><audio_37>pretty straightforward easy to implement if you start with this slotted page organization and you had extra fields that you designed in the first place with this begin end time stamp to keep track of all these pointers there is a interesting question</transcript> record has the same schema all of them now have this additional what is shown as a pointer but that pointer is like that big and end timestamp and bunch of other information that's associated with it so you'll just create the new version and then you'll maintain the pointers internally right so you can think of it as your slotted page organization being used to create a file and in that file you have record IDs so a z is a record ID A1 is a different record ID but there's this extra field of pointer that chains them together and allows you to maintain your singly link list right pretty straightforward easy to implement if you start with this slotted page organization and you had extra fields that you designed in the first place with this begin end time stamp to keep track of all these pointers there is a interesting question


<transcript><audio_38>with the version chain is a singly link list the question is where does a new version sit is it at the end of the chain or is it at the beginning of the chain and you could decide the implementation and their pros and cons</transcript> with the version chain is a singly link list the question is where does a new version sit is it at the end of the chain or is it at the beginning of the chain and you could decide the implementation and their pros and cons


<transcript><audio_39>to that from a perspective of of what is easy to implement it's very easy to implement something that goes from oldest to the newest because you just create a new record as you were just doing up over here a new value new a value needs to be created a new tupple</transcript><transcript><audio_40>you just create it at the end and then uh connect everything back to it like that so oldest to newest is the easiest and that's what will get used as the mechanism in this last uh project assignment that you have in buub uh but</transcript><transcript><audio_41>obviously the downside of that is if I have to find the latest version I have to Traverse the chain and notice These Chains may actually be spread across multiple pages so you may have to go through multiple pages to go get a record ID that you're interested in</transcript><transcript><audio_42>okay all right the other way is newest to all this but that's more difficult to implement you know it's kind of logically you have to create the new record move everything uh around to get that uh the change to work in the reverse Direction and you'll see there's</transcript><transcript><audio_43>a subtle Point associated with both of these as to how does an index refer to the record ID right so if I create a three version uh chain for the record a as you were saw in the slides a little while back it's like I have an index</transcript> to that from a perspective of of what is easy to implement it's very easy to implement something that goes from oldest to the newest because you just create a new record as you were just doing up over here a new value new a value needs to be created a new tupple you just create it at the end and then uh connect everything back to it like that so oldest to newest is the easiest and that's what will get used as the mechanism in this last uh project assignment that you have in buub uh but obviously the downside of that is if I have to find the latest version I have to Traverse the chain and notice These Chains may actually be spread across multiple pages so you may have to go through multiple pages to go get a record ID that you're interested in okay all right the other way is newest to all this but that's more difficult to implement you know it's kind of logically you have to create the new record move everything uh around to get that uh the change to work in the reverse Direction and you'll see there's a subtle Point associated with both of these as to how does an index refer to the record ID right so if I create a three version uh chain for the record a as you were saw in the slides a little while back it's like I have an index


<transcript><audio_44>which one does it point to and the implications of this oldest to newest and newest to oldest will become a little bit more apparent as we go through some of that okay but there are two ways that's all we need to know of organizing this list and it can go from</transcript><transcript><audio_45>one uh forward to backward or the other way around all the like the later no to point to the first one yeah so are you yeah in the newest to oldest uh the question is are how are you updating</transcript><transcript><audio_46>that so imagine uh this is you can just think of it as a singly link list and so here is a single link list maintained in a pendon storage going from oldest to newest if it's the other way around you'll just have to flip the pointers around so there's more pointer management stuff that you'll end up</transcript><transcript><audio_47>having to do you create a new record that becomes the head and what does it mean to be at the head it means that's what someone else refers to if they're coming from outside so it'll become clear in a second as we get to the index spot right the only thing you need to know is that yeah I can when I get to a</transcript> which one does it point to and the implications of this oldest to newest and newest to oldest will become a little bit more apparent as we go through some of that okay but there are two ways that's all we need to know of organizing this list and it can go from one uh forward to backward or the other way around all the like the later no to point to the first one yeah so are you yeah in the newest to oldest uh the question is are how are you updating that so imagine uh this is you can just think of it as a singly link list and so here is a single link list maintained in a pendon storage going from oldest to newest if it's the other way around you'll just have to flip the pointers around so there's more pointer management stuff that you'll end up having to do you create a new record that becomes the head and what does it mean to be at the head it means that's what someone else refers to if they're coming from outside so it'll become clear in a second as we get to the index spot right the only thing you need to know is that yeah I can when I get to a


<transcript><audio_48>record from somewhere else most often from an index do I land on the oldest or the newest and depending on the scheme to find my newest record I have to chase down the chain if it's newest to oldest if I'm only looking for new stuff which</transcript><transcript><audio_49>is often the most common pattern I'll find it right away I don't have to access a chain and these chains can span across pages so it can be very expensive to chase this chain down okay all right second type of storage is time travel</transcript> record from somewhere else most often from an index do I land on the oldest or the newest and depending on the scheme to find my newest record I have to chase down the chain if it's newest to oldest if I'm only looking for new stuff which is often the most common pattern I'll find it right away I don't have to access a chain and these chains can span across pages so it can be very expensive to chase this chain down okay all right second type of storage is time travel


<transcript><audio_50>and this is systems that globbed in an mvcc system when they didn't have one before will use something like that which requires the least disruptive change to your existing storage structure so here if I have a record and I create a new record A2 it will get put</transcript><transcript><audio_51>in the main table and then I've got a time travel table and in that time travel table I'll put the old values and the old values keep will keep getting accumulated in that second table so now if I need if I'm a reader and I'm only</transcript> and this is systems that globbed in an mvcc system when they didn't have one before will use something like that which requires the least disruptive change to your existing storage structure so here if I have a record and I create a new record A2 it will get put in the main table and then I've got a time travel table and in that time travel table I'll put the old values and the old values keep will keep getting accumulated in that second table so now if I need if I'm a reader and I'm only


<transcript><audio_52>allowed to read A2 I will start from the main table go down to a second table which is a different file so obviously that's more expensive that way you won't ever get locality uh to chase down that you're Crossing table boundaries but it doesn't disrupt the main table which</transcript><transcript><audio_53>just has a single copy the most recent copy and uh as you can see over here okay and that most recent copy may not be the one that eventually gets committed we'll start talking about the recovery protocols a little bit but it's the most recent copy in the chain at</transcript><transcript><audio_54>exists at that point in time if you use time travel storage you have to uh if you do time travel storage it implicitly gives you newest to all this yes that's observation that's correct and basically</transcript> allowed to read A2 I will start from the main table go down to a second table which is a different file so obviously that's more expensive that way you won't ever get locality uh to chase down that you're Crossing table boundaries but it doesn't disrupt the main table which just has a single copy the most recent copy and uh as you can see over here okay and that most recent copy may not be the one that eventually gets committed we'll start talking about the recovery protocols a little bit but it's the most recent copy in the chain at exists at that point in time if you use time travel storage you have to uh if you do time travel storage it implicitly gives you newest to all this yes that's observation that's correct and basically


<transcript><audio_55>it's done as you can imagine if you had a 30-year-old database system and all of a sudden mbcc is the way you get more concery especially for the readers this is the least disruptive change you would make to your system okay because your indices can keep pointing to the lasater copy in the</transcript><transcript><audio_56>main table all right and the record ID hasn't changed it still is in the same slot in the same page Why is it implicitly newest like I gu head sort of table the newest is there yeah</transcript> it's done as you can imagine if you had a 30-year-old database system and all of a sudden mbcc is the way you get more concery especially for the readers this is the least disruptive change you would make to your system okay because your indices can keep pointing to the lasater copy in the main table all right and the record ID hasn't changed it still is in the same slot in the same page Why is it implicitly newest like I gu head sort of table the newest is there yeah


<transcript><audio_57>so yeah and so here in this diagram as you can see it's a little weird A2 is the oldest which is pointing to A1 which is uh which is not necessarily the uh old the newest but and you can flip it around and switch it around in a</transcript><transcript><audio_58>different way so it's it can be hybrid between newest to oldest uh and oldest to newest but the newest copy is always going to be in the main table through which you start the access and that's the main point A2 is the newest one but you can</transcript><transcript><audio_59>imagine a weird scheme in which A2 is in the main table but that points to something else that you know if the chain will become A1 A2 A3 and A3 in the main table you could say yeah A3 points to A1 from the main table to the time travel table you can come up with</transcript> so yeah and so here in this diagram as you can see it's a little weird A2 is the oldest which is pointing to A1 which is uh which is not necessarily the uh old the newest but and you can flip it around and switch it around in a different way so it's it can be hybrid between newest to oldest uh and oldest to newest but the newest copy is always going to be in the main table through which you start the access and that's the main point A2 is the newest one but you can imagine a weird scheme in which A2 is in the main table but that points to something else that you know if the chain will become A1 A2 A3 and A3 in the main table you could say yeah A3 points to A1 from the main table to the time travel table you can come up with


<transcript><audio_60>schemes like that to be not purely newest to oldest but if you had to say which one it sort of makes more sense it's like the main table has a newest stuff right which is where you're starting your access from so you can do all kinds of fiber and with all of this stuff that's I'm going to speed through it a little bit you can find like 50 different schemes and there's a</transcript><transcript><audio_61>full-fledged paper which I'll leave you with because we could spend like four weeks just talking about mbcc all right so Master version in this case is going to be get written in</transcript><transcript><audio_62>place all right so now let's go on and so here that's just showing that the master is getting rid the preferred way is this Delta storage so far all of the storage if I made change to a single column I was actually making a full copy of the record and the record may have</transcript><transcript><audio_63>hundreds of columns so if I just changed one I'm doing everything uh I'm making a large amount of copy where the diff is really small so Delta storage is essentially that keyword if and I'll only take the value the column that I'm</transcript> schemes like that to be not purely newest to oldest but if you had to say which one it sort of makes more sense it's like the main table has a newest stuff right which is where you're starting your access from so you can do all kinds of fiber and with all of this stuff that's I'm going to speed through it a little bit you can find like 50 different schemes and there's a full-fledged paper which I'll leave you with because we could spend like four weeks just talking about mbcc all right so Master version in this case is going to be get written in place all right so now let's go on and so here that's just showing that the master is getting rid the preferred way is this Delta storage so far all of the storage if I made change to a single column I was actually making a full copy of the record and the record may have hundreds of columns so if I just changed one I'm doing everything uh I'm making a large amount of copy where the diff is really small so Delta storage is essentially that keyword if and I'll only take the value the column that I'm


<transcript><audio_64>changing which could be more than one in a given record depending upon what that update query is and I'm going to store in Delta storage uh that the value that has changed and then keep track of that through uh through a pointer right so</transcript><transcript><audio_65>essentially the big difference between the Delta storage and the other methods is here you'll just keep track of just the value that has changed and now this can get complicated to because there could be multiple values in a column that has changed as you can imagine you</transcript><transcript><audio_66>can generalize that stuff to only keeping track of the change that you've made so uh I noticed that whether you use data storage or time travel you always keep a copy of the most of value</transcript><transcript><audio_67>in the in the other uh do you add it when you update the main table or do you add it you add it like after you update the main yeah yeah yeah so you're asking uh when do I make a change in the main</transcript> changing which could be more than one in a given record depending upon what that update query is and I'm going to store in Delta storage uh that the value that has changed and then keep track of that through uh through a pointer right so essentially the big difference between the Delta storage and the other methods is here you'll just keep track of just the value that has changed and now this can get complicated to because there could be multiple values in a column that has changed as you can imagine you can generalize that stuff to only keeping track of the change that you've made so uh I noticed that whether you use data storage or time travel you always keep a copy of the most of value in the in the other uh do you add it when you update the main table or do you add it you add it like after you update the main yeah yeah yeah so you're asking uh when do I make a change in the main


<transcript><audio_68>table versus the Delta storage table so again there's going to be a bunch of details but most of the time what you going to do is you take the copy put it over there then update the value in there you'll move the old one stuff out to the whereas your backup storage or whether the chain storage before you</transcript><transcript><audio_69>make that because you have to remember that and you have to make that copy before because what happens if there's a crash between you making those changes if both the values are the same they'll have the same Tim stamp you can go clean some of that stuff if you need it to but</transcript> table versus the Delta storage table so again there's going to be a bunch of details but most of the time what you going to do is you take the copy put it over there then update the value in there you'll move the old one stuff out to the whereas your backup storage or whether the chain storage before you make that because you have to remember that and you have to make that copy before because what happens if there's a crash between you making those changes if both the values are the same they'll have the same Tim stamp you can go clean some of that stuff if you need it to but


<transcript><audio_70>if A2 in the main table doesn't get updated to A3 then theoretically there wouldn't be yeah yeah yeah and so you're getting into a little bit of this recovery protocols and the fact that sometimes you may want to undo some change because something</transcript><transcript><audio_71>Abed absolutely you all of that will happen with versioning to and we'll get to what we'll do like literally what we'll do is before we go and along with the changes that we make we'll also write what we are going to talk about next which is log records to maintain</transcript><transcript><audio_72>the change and then depending upon how you're using those log records you could do one of different schemes but we record stuff to undo things so all you need to know now is yeah bad things can happen while you're making these changes that's the recovery stuff we'll start talking about where we'll keep track of</transcript><transcript><audio_73>logs to to uh be able to back ourselves out of some unsafe state if we started to make this change and things crashed in the middle what the transaction AB it right yeah good question all right so Delta storage is that transactions can recreate in all of these things</transcript> if A2 in the main table doesn't get updated to A3 then theoretically there wouldn't be yeah yeah yeah and so you're getting into a little bit of this recovery protocols and the fact that sometimes you may want to undo some change because something Abed absolutely you all of that will happen with versioning to and we'll get to what we'll do like literally what we'll do is before we go and along with the changes that we make we'll also write what we are going to talk about next which is log records to maintain the change and then depending upon how you're using those log records you could do one of different schemes but we record stuff to undo things so all you need to know now is yeah bad things can happen while you're making these changes that's the recovery stuff we'll start talking about where we'll keep track of logs to to uh be able to back ourselves out of some unsafe state if we started to make this change and things crashed in the middle what the transaction AB it right yeah good question all right so Delta storage is that transactions can recreate in all of these things


<transcript><audio_74>transactions can recreate old values in the Tuple based schemes they just go to the tle and they get the whole thing here you can create old versions by applying the delta in the reverse order because it's creating it's keeping track of the Delta right even though it's not</transcript><transcript><audio_75>quite shown value is being changed it it'll say uh it might have some sort of a Delta associated with it and we'll talk about that from the logging perspective too but that notion of Delta is effectively the same now the minute you have versions</transcript><transcript><audio_76>you also need to do garbage collection and because as you keep building the version chains lots of updates let's say happen to a record at some point the version chains will build up new readers are coming in they have a new Tim stamp no one's reading no reader has a</transcript><transcript><audio_77>timestamp that was really really old so we need to clean up those versions okay and so uh there are again multiple ways in which you can go do this and we have to look for these expired versions and then we have to decide when it is safe</transcript> transactions can recreate old values in the Tuple based schemes they just go to the tle and they get the whole thing here you can create old versions by applying the delta in the reverse order because it's creating it's keeping track of the Delta right even though it's not quite shown value is being changed it it'll say uh it might have some sort of a Delta associated with it and we'll talk about that from the logging perspective too but that notion of Delta is effectively the same now the minute you have versions you also need to do garbage collection and because as you keep building the version chains lots of updates let's say happen to a record at some point the version chains will build up new readers are coming in they have a new Tim stamp no one's reading no reader has a timestamp that was really really old so we need to clean up those versions okay and so uh there are again multiple ways in which you can go do this and we have to look for these expired versions and then we have to decide when it is safe


<transcript><audio_78>to go and Rec claim that now when it is safe to do this expired version is typically you're going to say uh I have a some sort of transaction number or timestamp that I'm assigning to transactions and I kind of know what's the oldest transaction that is running</transcript><transcript><audio_79>in the system anything older than that I don't need so you have a way to go and figure out what portion of the tail of this version chain you can throw away okay now the question is when you have this type of garbage collection that you</transcript><transcript><audio_80>need now you've determined what you can throw away when do you go about doing that again there are two approaches you can do it at a topple level where you're going through the chains and I'm just going to assume everything is at a tle level the same thing will generalize if it's not and there are two methods for</transcript><transcript><audio_81>the topple level background vacuuming where a background thread does this Cooperative cleaning is kind of where you bust your own tables right so where you see something wrong you go fix it that's the second approach and then there's something to do at the transaction level so let's go through</transcript><transcript><audio_82>each of those techniques the tble level GC is pretty straightforward I've got got some background vacuuming thread that starts up every once in a while it goes and looks through all the place where the versions are right depending</transcript> to go and Rec claim that now when it is safe to do this expired version is typically you're going to say uh I have a some sort of transaction number or timestamp that I'm assigning to transactions and I kind of know what's the oldest transaction that is running in the system anything older than that I don't need so you have a way to go and figure out what portion of the tail of this version chain you can throw away okay now the question is when you have this type of garbage collection that you need now you've determined what you can throw away when do you go about doing that again there are two approaches you can do it at a topple level where you're going through the chains and I'm just going to assume everything is at a tle level the same thing will generalize if it's not and there are two methods for the topple level background vacuuming where a background thread does this Cooperative cleaning is kind of where you bust your own tables right so where you see something wrong you go fix it that's the second approach and then there's something to do at the transaction level so let's go through each of those techniques the tble level GC is pretty straightforward I've got got some background vacuuming thread that starts up every once in a while it goes and looks through all the place where the versions are right depending


<transcript><audio_83>on the scheme it's going to be in a different place and then it's going to say I know what the latest transactions are so as these things are getting changed you end up with a place where you say at this point I can determine that A1 the 100 uh anything with a</transcript><transcript><audio_84>version of 100 is no longer needed because I know all the transactions that are in the system no one needs a value older than 100 and I can basically go and start to vacuum those out and remove that now remember these version chains could be long they could be spread</transcript><transcript><audio_85>across different pages and things like that so this vacuuming process can be pretty expensive and it's making extensive changes to that entire database uh which you know obviously is not just one file one table but all the tables could have had this type of</transcript><transcript><audio_86>version management that is needed one optimization you could do in Stu saying I'm always going to scan all the file files from start to the end from the first page to the last page every time some update happens to a page you'll keep track of a simple metadata which</transcript> on the scheme it's going to be in a different place and then it's going to say I know what the latest transactions are so as these things are getting changed you end up with a place where you say at this point I can determine that A1 the 100 uh anything with a version of 100 is no longer needed because I know all the transactions that are in the system no one needs a value older than 100 and I can basically go and start to vacuum those out and remove that now remember these version chains could be long they could be spread across different pages and things like that so this vacuuming process can be pretty expensive and it's making extensive changes to that entire database uh which you know obviously is not just one file one table but all the tables could have had this type of version management that is needed one optimization you could do in Stu saying I'm always going to scan all the file files from start to the end from the first page to the last page every time some update happens to a page you'll keep track of a simple metadata which


<transcript><audio_87>might have one bit per page that marks the page as dirty so you you've got a billion pages in your database and only thousand of them were touched the vacuum process doesn't need to go and read the billion pages to determine have you been touched right is there a version tail in</transcript><transcript><audio_88>their page that I need to clean up it'll just go and look at the pages that are the ones that need this cleanup and only go and change those okay so that's an obvious optimization you would build especially if you're doing this on a large scale Cooperative</transcript> might have one bit per page that marks the page as dirty so you you've got a billion pages in your database and only thousand of them were touched the vacuum process doesn't need to go and read the billion pages to determine have you been touched right is there a version tail in their page that I need to clean up it'll just go and look at the pages that are the ones that need this cleanup and only go and change those okay so that's an obvious optimization you would build especially if you're doing this on a large scale Cooperative


<transcript><audio_89>cleaning is you know if you don't if you go to a sit-down restaurant they'll bring all the uh food for you and take away the plates from you but if you go to a place where you're busing it yourself you'll go to the counter pick up your food and uh clean up your tray it's kind of like that uh with</transcript> cleaning is you know if you don't if you go to a sit-down restaurant they'll bring all the uh food for you and take away the plates from you but if you go to a place where you're busing it yourself you'll go to the counter pick up your food and uh clean up your tray it's kind of like that uh with


<transcript><audio_90>Cooperative cleaning as you do the work the workers as they do their work will identify if they see something that needs to be cleaned up and they will do do the cleanup themselves and sometimes it's like when do you do the cleanup while you're doing the work or after transaction commits those are other</transcript><transcript><audio_91>choices that we can make we'll ignore those comments but the difference is that you don't have a separate background thread that's doing that you basically do a paid forward style of work where imagine I am a transaction that wants to get a value a I use an</transcript><transcript><audio_92>index to get to that that index now gets me to the a record but that a record imagine the chaining technique that we use is oldest to newest so it brings me to the head of the oldest I know what my time stamp is right so I know I don't need that I need to keep chasing it down</transcript><transcript><audio_93>till I get to the one that I should be reading but as I do that since I'm already chasing down the version chain I'm bringing the pages into disk if I need to might as well go and help uh</transcript> Cooperative cleaning as you do the work the workers as they do their work will identify if they see something that needs to be cleaned up and they will do do the cleanup themselves and sometimes it's like when do you do the cleanup while you're doing the work or after transaction commits those are other choices that we can make we'll ignore those comments but the difference is that you don't have a separate background thread that's doing that you basically do a paid forward style of work where imagine I am a transaction that wants to get a value a I use an index to get to that that index now gets me to the a record but that a record imagine the chaining technique that we use is oldest to newest so it brings me to the head of the oldest I know what my time stamp is right so I know I don't need that I need to keep chasing it down till I get to the one that I should be reading but as I do that since I'm already chasing down the version chain I'm bringing the pages into disk if I need to might as well go and help uh


<transcript><audio_94>clean up till I get to the uh the value the record that I need to read which may by the way not be at the end of the chain right because it may be depending on what my transaction number is I I should be reading that value A2 and there may be newer versions behind it but whenever I need to stop I'll stop</transcript> clean up till I get to the uh the value the record that I need to read which may by the way not be at the end of the chain right because it may be depending on what my transaction number is I I should be reading that value A2 and there may be newer versions behind it but whenever I need to stop I'll stop


<transcript><audio_95>and clean up stuff before me I this is all situations where let's say you have some uh some chain that's really long that you never access yes so you might just that that might never be cleaned out yeah so it can even happen that if</transcript><transcript><audio_96>I've got a chain that is really long and that uh record is no longer access it was very hot in 2020 and a billion uh you know Taylor Swift ticket counter and the Taylor Swift ticket counter for a venue was really popular it had a</transcript><transcript><audio_97>million version chains million entries built up in the version chain now she has a new uh venue and no one's going and cleaning that up and that could happen you could ask a more philosophical question is why did the chains build up if everyone is cleaning up on the fly but we'll defer that sometimes they may be deferring that I</transcript><transcript><audio_98>thought that's what you're going to ask but that that can that can happen Okay and again as I said there are like so many million ways of combining these things in different ways that all you need to know is that there are these different schemes to do it and there are</transcript> and clean up stuff before me I this is all situations where let's say you have some uh some chain that's really long that you never access yes so you might just that that might never be cleaned out yeah so it can even happen that if I've got a chain that is really long and that uh record is no longer access it was very hot in 2020 and a billion uh you know Taylor Swift ticket counter and the Taylor Swift ticket counter for a venue was really popular it had a million version chains million entries built up in the version chain now she has a new uh venue and no one's going and cleaning that up and that could happen you could ask a more philosophical question is why did the chains build up if everyone is cleaning up on the fly but we'll defer that sometimes they may be deferring that I thought that's what you're going to ask but that that can that can happen Okay and again as I said there are like so many million ways of combining these things in different ways that all you need to know is that there are these different schemes to do it and there are


<transcript><audio_99>exclusive Exquisite large number of ways in which you could combine all of this but I do want to get to the indexing part which is important but before we do that remember two slides ago we said garbage collection tupple level and transaction level right so we finished</transcript><transcript><audio_100>the tupple level stuff now let's go to the transaction level stuff the transaction level stuff is very is different every transaction is going to keep track of all the stuff that it is doing including things that it is now</transcript><transcript><audio_101>outdating because it created new versions a transaction knows right when it has added something to the version change so in this approach as a transaction proceeds it's doing an update uh and then the second operation comes in it's creating these old</transcript><transcript><audio_102>versions the basic thing is when a transaction is done it knows I created two versions one for record a one for record B and at that point a vacuum process can be handed over those versions and now</transcript> exclusive Exquisite large number of ways in which you could combine all of this but I do want to get to the indexing part which is important but before we do that remember two slides ago we said garbage collection tupple level and transaction level right so we finished the tupple level stuff now let's go to the transaction level stuff the transaction level stuff is very is different every transaction is going to keep track of all the stuff that it is doing including things that it is now outdating because it created new versions a transaction knows right when it has added something to the version change so in this approach as a transaction proceeds it's doing an update uh and then the second operation comes in it's creating these old versions the basic thing is when a transaction is done it knows I created two versions one for record a one for record B and at that point a vacuum process can be handed over those versions and now


<transcript><audio_103>that can determine saying my uh oldest version I need is 10 onwards if all of these are less than 10 I can go clean it up right so the vacuum process doesn't have to go through each and every page even with that dirty page marker to determine where these versions are</transcript><transcript><audio_104>transactions just hand it over to you so the and those are the things that need to get cleaned up okay so just a different way of doing that uh and clearing up the versions now this I want us to spend a little bit of time on okay</transcript><transcript><audio_105>the rest of this uh uh uh material I'll just basically leave it in slides and let you uh worry about it in other words index management is stuff up to all of this material material for exam questions but the last part I'm going to skim over and you don't need to worry</transcript><transcript><audio_106>about so pay attention now all right so indexes point to object IDs to record IDs right imagine you have a slotted page organization it's going to say page number and the slot number now if I've got a primary key index and I could be</transcript> that can determine saying my uh oldest version I need is 10 onwards if all of these are less than 10 I can go clean it up right so the vacuum process doesn't have to go through each and every page even with that dirty page marker to determine where these versions are transactions just hand it over to you so the and those are the things that need to get cleaned up okay so just a different way of doing that uh and clearing up the versions now this I want us to spend a little bit of time on okay the rest of this uh uh uh material I'll just basically leave it in slides and let you uh worry about it in other words index management is stuff up to all of this material material for exam questions but the last part I'm going to skim over and you don't need to worry about so pay attention now all right so indexes point to object IDs to record IDs right imagine you have a slotted page organization it's going to say page number and the slot number now if I've got a primary key index and I could be


<transcript><audio_107>updating the primary key and I'm making changes to that I now need to figure out how I'm going to keep track of that version change and version change stuff we can keep track of the techniques we talked about but now I have to go and update that primary key now it gets very</transcript><transcript><audio_108>tricky so often what happens when you're doing this versioning type of mechanism for primary key updates you'll basically treat it like a delete followed by an insert it makes the semantics clean and you won't have that version chain developed right because you delete</transcript><transcript><audio_109>everything's gone and do an insert okay the problem becomes more interesting with secondary keys because they're more complicated right in some sense the primary key controls the record right because you're accessing the record the record ID and the primary key in many ways are analogous so you can do these</transcript><transcript><audio_110>things like insert delete but for the second Ries you can't do that right there's a second riy and there's a very famous incident that happened at Uber a few years ago where they used to be on my SQL that had a good way of doing</transcript><transcript><audio_111>secondary indexes and postest doesn't have a good way so it is multiversion both of them are multiversion and as a result if you have lots of updates happening in your system the performance will go down quite poorly unless you have the good way of doing secondary indices so we'll talk about the good and</transcript> updating the primary key and I'm making changes to that I now need to figure out how I'm going to keep track of that version change and version change stuff we can keep track of the techniques we talked about but now I have to go and update that primary key now it gets very tricky so often what happens when you're doing this versioning type of mechanism for primary key updates you'll basically treat it like a delete followed by an insert it makes the semantics clean and you won't have that version chain developed right because you delete everything's gone and do an insert okay the problem becomes more interesting with secondary keys because they're more complicated right in some sense the primary key controls the record right because you're accessing the record the record ID and the primary key in many ways are analogous so you can do these things like insert delete but for the second Ries you can't do that right there's a second riy and there's a very famous incident that happened at Uber a few years ago where they used to be on my SQL that had a good way of doing secondary indexes and postest doesn't have a good way so it is multiversion both of them are multiversion and as a result if you have lots of updates happening in your system the performance will go down quite poorly unless you have the good way of doing secondary indices so we'll talk about the good and


<transcript><audio_112>the bad way and so Uber went back and changed it again to postest from postest to my SQL because they realized that this was a problem so you know they hadn't taken this class uh so secondary Ines are going to point to logical</transcript><transcript><audio_113>pointers inherence is going to be the problem is like those are think of it as surrogate pointers they can't control the object they can't willly do this delete followed by an insert uh technique and there two approaches one is a logical pointer and a physical pointer so just look at it with a</transcript><transcript><audio_114>diagram I've got a version chain A4 to A1 and if I have uh appended stuff to it and let's assume this is newest to oldest okay and same things will apply for other schemes but newest to Old this is the is the interesting one I say get</transcript><transcript><audio_115>this value a from this primary key and I will go I'll get the record ID right that's what the index is going to have the record ID and I'll go find the record A4 which is the fourth version of the record a but it's a physical record that I'm going to get I'm going to get a</transcript> the bad way and so Uber went back and changed it again to postest from postest to my SQL because they realized that this was a problem so you know they hadn't taken this class uh so secondary Ines are going to point to logical pointers inherence is going to be the problem is like those are think of it as surrogate pointers they can't control the object they can't willly do this delete followed by an insert uh technique and there two approaches one is a logical pointer and a physical pointer so just look at it with a diagram I've got a version chain A4 to A1 and if I have uh appended stuff to it and let's assume this is newest to oldest okay and same things will apply for other schemes but newest to Old this is the is the interesting one I say get this value a from this primary key and I will go I'll get the record ID right that's what the index is going to have the record ID and I'll go find the record A4 which is the fourth version of the record a but it's a physical record that I'm going to get I'm going to get a


<transcript><audio_116>physical record ID and I can locate A4 so far everything is good no problem right now you can see if the primary key is getting updated if I delete and insert it then I don't have to worry about a bunch of this the secondary indices however if I say get something</transcript><transcript><audio_117>which has to get to this record a the record ID is going to point to A4 and that's okay except I could have multiple secondary indices on the record a you know it has five columns I could have built a secondary index on each of those five</transcript> physical record ID and I can locate A4 so far everything is good no problem right now you can see if the primary key is getting updated if I delete and insert it then I don't have to worry about a bunch of this the secondary indices however if I say get something which has to get to this record a the record ID is going to point to A4 and that's okay except I could have multiple secondary indices on the record a you know it has five columns I could have built a secondary index on each of those five


<transcript><audio_118>columns now each of these secondary indexes is is going to point to that first record for old the newest version in this version chain again so far everything is good nothing nothing bad</transcript><transcript><audio_119>has happened so far now you start to get into some uh issues which is if I have to go and update this value and create a new version A5 what's going to happen I'll have to</transcript><transcript><audio_120>update A5 I'll get a new record in a page fix the version chain using these version management schemes that we discussed maybe you have Delta storage now I have to go and update each of the secondary indices to have the record IDs</transcript><transcript><audio_121>point to the new A5 which is a lot of updates so One update to the record will cause every secondary index to be updated right so you can see how this starts to become a huge problem right</transcript> columns now each of these secondary indexes is is going to point to that first record for old the newest version in this version chain again so far everything is good nothing nothing bad has happened so far now you start to get into some uh issues which is if I have to go and update this value and create a new version A5 what's going to happen I'll have to update A5 I'll get a new record in a page fix the version chain using these version management schemes that we discussed maybe you have Delta storage now I have to go and update each of the secondary indices to have the record IDs point to the new A5 which is a lot of updates so One update to the record will cause every secondary index to be updated right so you can see how this starts to become a huge problem right


<transcript><audio_122>it I already showed you yeah good good good but you paying attention that's great see in computer science indirection is a very powerful technique you know 50% of problems can be solved with indirection right uh so instead of secondary indices</transcript><transcript><audio_123>pointing to the record IDs what you would do is you would say because the primary key is kind of like a record ID the secondary index will say I'm just going to point to the primary key which does mean that when I'm accessing a record through the secondary key I have to go to the primary key and then get</transcript><transcript><audio_124>the record ID one extra but if it's if it's hot then probably the primary key index is already in the buffer pool so it's not too bad but what that gives me is that now if I change a record all I do is make the changes in the primary index the secondary index</transcript><transcript><audio_125>stuff doesn't have to change so a single update to a single column think about in Uber's case if they changed the rate for uh uh taxi service and that's changing all the time and that's one field that was changed in a record now all the indices have to change that just causes</transcript> it I already showed you yeah good good good but you paying attention that's great see in computer science indirection is a very powerful technique you know 50% of problems can be solved with indirection right uh so instead of secondary indices pointing to the record IDs what you would do is you would say because the primary key is kind of like a record ID the secondary index will say I'm just going to point to the primary key which does mean that when I'm accessing a record through the secondary key I have to go to the primary key and then get the record ID one extra but if it's if it's hot then probably the primary key index is already in the buffer pool so it's not too bad but what that gives me is that now if I change a record all I do is make the changes in the primary index the secondary index stuff doesn't have to change so a single update to a single column think about in Uber's case if they changed the rate for uh uh taxi service and that's changing all the time and that's one field that was changed in a record now all the indices have to change that just causes


<transcript><audio_126>the massive problem and Hance that huge performance problem okay there's another way to do the IND Direction which is to say what if you had a global structure that converted Tuple ID to the to some sort of an address and everyone went through that into the primary key you</transcript><transcript><audio_127>could do that but no one does that because the primary key uh index is essentially that there have been proposals that talk about doing that because hey does this mean if I have to use this more fancy scheme which is better for performance with the secondary key indices that I must have a</transcript><transcript><audio_128>primary key index the answer is yes the primary key index is typically always built because that's how the system enforces primary key constraint so you can assume especially in that environment you have that so that scheme is basically the preferred scheme to do</transcript><transcript><audio_129>it okay so if you're doing potioning with just about every system does you have to be careful as to what your secondary indices point to all</transcript><transcript><audio_130>question it always Remains the Same for the newest yeah location always the same yeah then how does all of this come into this like this only works if you have theend only version yeah so this works</transcript> the massive problem and Hance that huge performance problem okay there's another way to do the IND Direction which is to say what if you had a global structure that converted Tuple ID to the to some sort of an address and everyone went through that into the primary key you could do that but no one does that because the primary key uh index is essentially that there have been proposals that talk about doing that because hey does this mean if I have to use this more fancy scheme which is better for performance with the secondary key indices that I must have a primary key index the answer is yes the primary key index is typically always built because that's how the system enforces primary key constraint so you can assume especially in that environment you have that so that scheme is basically the preferred scheme to do it okay so if you're doing potioning with just about every system does you have to be careful as to what your secondary indices point to all question it always Remains the Same for the newest yeah location always the same yeah then how does all of this come into this like this only works if you have theend only version yeah so this works


<transcript><audio_131>well well when you have newest to oldest so if my newest comes in an A5 it's got a new record ID right so in this case it'll have a new record ID that I need to point to you could have said if I had oldest to newest I wouldn't have this</transcript><transcript><audio_132>which primary key points does it Point towards the table index like where in the table do you have the versioning stuff or do does it point to the actual physical location on dis of the yeah so it depends on the scheme if I newest to oldest every index is going to point to</transcript> well well when you have newest to oldest so if my newest comes in an A5 it's got a new record ID right so in this case it'll have a new record ID that I need to point to you could have said if I had oldest to newest I wouldn't have this which primary key points does it Point towards the table index like where in the table do you have the versioning stuff or do does it point to the actual physical location on dis of the yeah so it depends on the scheme if I newest to oldest every index is going to point to


<transcript><audio_133>the newest head of the link okay and so he said look for primary key updates we're going to assume that it's delete followed by an insert so ignore that case right we swept It Away by a different type of implementation so in this case now I've got uh newest to</transcript><transcript><audio_134>oldest and the version chain has built up A4 is the newest which is being referred to by all the indices right so now if I add let's say a new A5 version then all the secondary Industries have to point to A5 because you know they have to see the whole</transcript> the newest head of the link okay and so he said look for primary key updates we're going to assume that it's delete followed by an insert so ignore that case right we swept It Away by a different type of implementation so in this case now I've got uh newest to oldest and the version chain has built up A4 is the newest which is being referred to by all the indices right so now if I add let's say a new A5 version then all the secondary Industries have to point to A5 because you know they have to see the whole


<transcript><audio_135>version chain to Traverse through it but in suppose the time Trav version remains in the same in the same position oh you saying in the time travel where I had my you're talking about the scheme number two where the main table had it yep so that's scario</transcript><transcript><audio_136>don't really the poin right they could continue pointing towards the new one yeah yeah yeah that's right so in that scheme and you're exactly right in the scheme where we had a separate time travel table that was the lowest lift for someone to go and get mvcc there</transcript><transcript><audio_137>they're pointing to that main table so the record ID hasn't moved it's the same page ID slot ID and this problem is is is simpler over there absolutely but as I said look there are all kinds of schemes that will go through that I don't want to spend an infinite amount of time in looking through all the</transcript> version chain to Traverse through it but in suppose the time Trav version remains in the same in the same position oh you saying in the time travel where I had my you're talking about the scheme number two where the main table had it yep so that's scario don't really the poin right they could continue pointing towards the new one yeah yeah yeah that's right so in that scheme and you're exactly right in the scheme where we had a separate time travel table that was the lowest lift for someone to go and get mvcc there they're pointing to that main table so the record ID hasn't moved it's the same page ID slot ID and this problem is is is simpler over there absolutely but as I said look there are all kinds of schemes that will go through that I don't want to spend an infinite amount of time in looking through all the


<transcript><audio_138>combinations I will leave you with a paper that talks about a lot of these combinations okay and then what are the different pros and cons but I'll take that final question before we move on so you said for time travel you don't</transcript><transcript><audio_139>but for Delta don't yeah and you know again it'll get into some other nuances even for time travel you can think about there's a version chain that's maintained but the head if it remains over there in the page ID and the record ID and it's updated in place then this problem doesn't happen if you have</transcript><transcript><audio_140>update in place and that's the head of that uh uh table place for ad you can come up with all kinds of schemes to avoid this problem but most systems will do some sort of version management that may end up</transcript><transcript><audio_141>having this problem just need you to be aware that if your record IDs are moving around then if you're using any sort of version management scheme then you're going to run into trouble like this okay okay good the rest of it from here</transcript><transcript><audio_142>onwards slide 27 onwards it's not going to be material for the exam so I'm just going to skip skim through it really fast uh because mvcc looks like the keys Canan be duplicate it turns out that even the primary keys if I'm using the</transcript><transcript><audio_143>scheme in which I'm updating a primary key and I have versions it'll kind of look like if you think about the implementation of B tree for typically for a primary key you would have implemented to say I can never have a duplicate key but in some sense the keys can have you may need duplicates for a</transcript> combinations I will leave you with a paper that talks about a lot of these combinations okay and then what are the different pros and cons but I'll take that final question before we move on so you said for time travel you don't but for Delta don't yeah and you know again it'll get into some other nuances even for time travel you can think about there's a version chain that's maintained but the head if it remains over there in the page ID and the record ID and it's updated in place then this problem doesn't happen if you have update in place and that's the head of that uh uh table place for ad you can come up with all kinds of schemes to avoid this problem but most systems will do some sort of version management that may end up having this problem just need you to be aware that if your record IDs are moving around then if you're using any sort of version management scheme then you're going to run into trouble like this okay okay good the rest of it from here onwards slide 27 onwards it's not going to be material for the exam so I'm just going to skip skim through it really fast uh because mvcc looks like the keys Canan be duplicate it turns out that even the primary keys if I'm using the scheme in which I'm updating a primary key and I have versions it'll kind of look like if you think about the implementation of B tree for typically for a primary key you would have implemented to say I can never have a duplicate key but in some sense the keys can have you may need duplicates for a


<transcript><audio_144>little bit of time okay so again as I said I'm not going to go into the details for it but let you look through that and you can read the chapter in the book that talks about that but you have to worry about uh those components there's also the issue of deletes again</transcript> little bit of time okay so again as I said I'm not going to go into the details for it but let you look through that and you can read the chapter in the book that talks about that but you have to worry about uh those components there's also the issue of deletes again


<transcript><audio_145>I'll let you read that by yourself but there are different ways to implement the delete function by keeping a delete flag or a tombstone flag this has to do with I've got a version chain and if I delete it uh there are different ways of deleting that and sometimes it's uh it's</transcript><transcript><audio_146>uh better to just Mark a record has been deleted and then eventually it can get uh cleaned up or you can have a tombstone uh based approach to basically clean up this version with in a in a two-step process so again the details for this not material for the exam but I</transcript><transcript><audio_147>uh encourage you to read the textbook but up to here the key problem I do want you to know and we may ask you questions about that okay uh last piece over here just for</transcript><transcript><audio_148>those of you who have more curiosity is the there's an explosion in the combinations that you can achieve with all of these different ways what's my garbage collection what's my index doing is it pointing to a physical uh record ID or is it pointing to a logical record ID that I can create in one of many</transcript> I'll let you read that by yourself but there are different ways to implement the delete function by keeping a delete flag or a tombstone flag this has to do with I've got a version chain and if I delete it uh there are different ways of deleting that and sometimes it's uh it's uh better to just Mark a record has been deleted and then eventually it can get uh cleaned up or you can have a tombstone uh based approach to basically clean up this version with in a in a two-step process so again the details for this not material for the exam but I uh encourage you to read the textbook but up to here the key problem I do want you to know and we may ask you questions about that okay uh last piece over here just for those of you who have more curiosity is the there's an explosion in the combinations that you can achieve with all of these different ways what's my garbage collection what's my index doing is it pointing to a physical uh record ID or is it pointing to a logical record ID that I can create in one of many