THAT PLUS I GUESS IT'S ACTUALLY STRAIGHTFORWARD TO SAY TO START WITH IS THIS A SMALL WORLD NETWORK AND THEN RUN IT ON THAT YOU WERE RUNNING THAT ON WHAT WE WHAT WAS IT LIKE HALF A MILLION PAGES SOMETHING LIKE THAT IS IT AND THAT WAS FOR HALF OF IT UH IF I REMEMBER OKAY OKAY THE WHOLE THING WAS AND WELL BUT DID YOU MANAGE TO TO DIVIDE THE LINK UH PROPERLY OR WELL WHAT I DID WITH THE LINK WELL BUT THIS IS NOT TH EXACTLY THE SAME SETUP YOU WOULD HAVE SO IN THIS SETUP THE WHOLE LINK STRUCTURE COULD BE KNOWN AND THE ONLY THING YOU WOU DON'T YOU W DON'T KNOW WOULD BE THE ASSIGNATION TO UH TO A CATEGORY IN MY SETUP IT WAS DIFFERENT BECAUSE I I SH I COULDN'T USE THE LINKS BETWEEN DIFFERENT SETS BUT THIS IS NOT I DON'T WANT TO MESS YOU UP WITH THAT IT'S NOT THE SAME KIND OF EXPERIMENT BUT YOU COULD JUST USE THE WHOLE CORPUS LIKE THIS AND AND FOR SOME OF THEM YOU ASSUME YOU DON'T KNOW THE THE CATEGORY ASSIGNMENT AND THEN YOU CAN RUN THIS SET OF EXPERIMENTS OVER WIKIPEDIA I MEAN I THINK OFF HAND THINKING ABOUT THIS MODEL IT WOULD HAVE TO WORK BETTER BECAUSE TWO THINGS IN THE SAME CATEGORY ARE GOING TO BE LINKED SO IT IT YEAH OKAY YEAH THAT'S I THINK THA IT'S GONNA BE MORE TRUE IN SOME NETWORKS AND IN SOME TASKS THAN OTHERS AND I THINK IN THE WIKIPEDIA IT'S PROBABLY GONNA BE IT'S A GOOD ONE TO TEST ON LIKE FOR EXAMPLE YOU CAN TAKE PROCEEDINGS AND ASSUME THE CATEGORIES ARE THE KEYWORDS OF THE OF THE D OF THE DOCUMENTS AND THEN IF YOU HAVE THE ARCHIVE OF A JOURNAL YOU CAN YOU CAN USE THE L THE LINKS BETWEEN UH THE CITATION LINKS SO IF YOU WANT TO HAVE A BIGGER STUFF SO WIKIPEDIA WOULD BE KIND OF VERY SMALL WITH LOTS OF ENGLI IN LINKS AND THEN IF YOU TAKE A JOURNAL YOU WILL HAVE LESS LINKS AND AND MORE DOCUMENTS AND THEN YOU WOULD BE LIKELY TO SAY THAT P. L. S. A. MIGHT BE BETTER OVER THAT BECAUSE YOU DON'T HAVE ENOUGH LINKS SO YOU YOU YOU CAN FIND EXAMPLES WHICH ARE NOT THE WEB I THINK IT'S GOOD TO WORK IN CLOSED SET WIKIPEDIA IS KIND OF CLOSED SET MA EVEN IF THERE'S LOTS OF OUTSIDE LINKS BUT THERE'S LOTS OF INSIDE LINKS YOU HAVE TO THINK AT AT AT AT AT AT AT DATABASE WHERE THERE'S LOT OF IN LINKS WIKIPEDIA MIGHT BE ONE CITATION FOR EXAMPLE IF IF YOU TAKE THE NIPS ARCH ARCHIVE WHICH IS AVAILABLE NIPS AUTHOR ALWAYS OTHER NIPS AUTHOR WHICH ARE OFTEN THEMSELVES ETCETERA WELL IT'S REPORTED I I WON'T SAY SO BUT BUT BUT BUT YEAH I THINK TH TH THIS KIND OF SETUP MAYBE WEB PAGES IT'S HARD TO FIND SETS OF WEB PAGES WHICH HAVE GOOD LINKS STUFF IF YOU START FROM ONE PAGE OR SOMETHING OKAY YEAH IT I'LL MAYBE I'LL HAVE A LOOK AT THE WIKIPEDIA STUFF ANYWAY I'M STUCK DOING SOMETHING ELSE FOR THE NEXT MONTH OR TWO ANYWAY SO UH THIS IS ALWAYS THE CASE UH THE THERE'S ANOTHER THING UM UH THIS WAS NEW BUT I HAVEN'T THOUGHT ABOUT THIS TOO MUCH BUT IT'S ON THE SAME LINES IN THAT UH HAVE HAVE YOU GUYS SEEN THIS UH CORRELATED TOPIC MODELS WORK CORRELATED TOPIC MODELS IT'S DAVID BLEI THE GUY THAT CAME UP WITH UH WITH L. D. A. HE'S GOT UH A NEW MODEL THAT'S IT'S PUBLISHED IN NIPS THIS YEAR BUT HIS CO AUTHOR PUT IT ON THEIR WEB SITE A MONTH OR TWO AGO SO UH YOU KNOW L. D. A. D YOU ALL KNOW L. D. A. OR YEAH UH OKAY SO A L. D. A. YOU BASICALLY DRAW A A DIRICHLET WHICH SAYS LIKE WHAT TYPE OF DOCUMENT THIS IS EFFECTIVELY AND THEN YOU DRAW A TOPIC GIVEN THAT DIRICHLET DISTRIBUTION AND THEN YOU DRAW WORDS GIVEN THAT ALRIGHT WELL THAT'S FOR FOR EVERY WORD AND THAT'S ONCE FOR THE DOCUMENT AND THAT'S YOU'VE GOT A PARAMETER COMING AT THIS GLOBALLY SO THAT THAT'S ROUGHLY WHAT THE MODEL IS PLUS OTHER LITTLE BITS BUT ANYWAY SO WHAT THEY DID WAS THEY THEY REWORKED THIS BECAUSE ONE OF THE PROBLEMS WITH L. D. A. AND I THINK P. L. S. A. HAS THIS PROBLEM AS WELL IN IS THAT UH THE MORE TOPICS YOU HAVE THE LESS UH EXPRESSIVE IT CAN BE BECAUSE THE TOPICS IT CLUSTER IT IT PUSHES A DOCUMENT IF IT IS PULLING A DOCUMENT INTO ONE TOPIC IT PUSHES ANOTHER TOPIC AWAY FROM THAT DOESN'T THAT MAKE SENSE SO THE TOPICS ARE NECESSARILY TRYING TO BE DIFFERENT FROM EACH OTHER SO THE MORE TOPICS YOU HAVE THE MORE TH THEY'RE SORT OF SQUEEZING INTO THE SPACE OF WHAT THINGS CAN BE ABOUT UH AND THINGS ARE KINDA PUSHED OUT SO ONCE YOU M GO OVER YOU TRY IT ONE CORPUS AND ONCE YOU GO OVER ABOUT A HUNDRED MAYBE TWO HUNDRED TOPICS UH THE THE NEW YOU ADD MORE TOPICS AND IT DOESN'T GET BETTER IN ANY WAY THE MODEL DOESN'T THE N NEW TOPICS ARE EFFECTIVELY GOING TO JUST BE GARBAGE UH ALRIGHT SO WHAT THEY DID WAS THEY THEY THEY STOPPED USING THE DIRICHLET AND UH THEY'RE STARTING TO USE UH A GAUSSIAN THERE SO IT IT IN TO SPARE THE DETAILS BASICALLY THEY'VE THEY'VE GOT A VARIABLE HERE I THINK THEY CALL IT ETA UH AND THEY FROM THIS GENERATE A DISTRIBUTION OF TOPICS IN SOME WAY BUT IT'S GENERATED FROM A GAUSSIAN SPACE FROM A FROM A A SPACE SO IT'S GIVEN A A MEAN AND COVARIANCE UH AND YEAH UH SO WHAT IT DOES THE THIS IS THE THE BIT THAT TOOK ME FOREVER TO UNDERSTAND ACTUALLY UH SAME MODEL BUT THIS IS GIVEN A GAUSSIAN SPACE IT MAPS THAT DOWN INTO THE SIMPLEX OF WHAT A TOPIC CAN BE ABOUT THE SO THE SIMPLEX BASICALLY IF YOU'VE GOT A SAY YOU'VE GOT TWO TOPICS NOW SAY I I SAY YOU GET THREE TOPICS THAT MAKES MORE EASIER TO DRAW UH FOR THIS IS THETA ONE THETA TWO THETA THREE THESE ARE THE PROBABILITIES THAT THAT A DOCUMENT IS ABOUT UH THIS THETA ONE IS THE PROBABILITY THAT THIS DOCUMENT IS ABOUT TOPIC ONE THETA TWO IS SO THESE GUYS ALL HAVE TO ADD UP TO ONE YEAH UH SO THAT BASICALLY MEANS THAT WHAT A DOCUMENT IS ABOUT LIES ON THIS SPACE BETWEEN THEM YEAH SO THIS IS THE SIMPLEX UH SO BASICALLY A DISTRIBUTION GENERATED BY THIS GUY IS BASICALLY A POINT ON THIS SIMPLEX SO WHAT THEY DO IS THEY'VE GOT A DISTRIBUTION THAT MAPS UH A GAUSSIAN AND SOME SPACE INTO A DISTRIBUTION ON THIS THING SO IT MAPS IT INTO GAUSSIAN IN THAT THE COOL THING ABOUT THAT IS THAT IF YOU'VE GOT TWO GAUSSIANS YOU CAN TELL HOW CLOSE THEY ARE SO YOU CAN NOW TELL HOW CLOSE TWO TOPIC DISTRIBUTIONS ARE RIGHT