THAT'S UM ONE THINGS THEY DID RIGHT AT THE BEGINNING IS THEY'VE GOT THIS BIG S STOP LIST AND THEY JUST IGNORE ALL OF THOSE THROUGHOUT THE EXPERIMENT SO WE NEED LIKE SEVERAL OF US NEED A DICTIONARY AM I THE ONLY ONE WHO NEEDS IT WITH FREQUENCIES AM I THE ONLY ONE WHO NEEDS IT WITH FREQUENCIES OR IT UH I THINK THAT'D BE USEFUL FOR ME AS WELL YEAH YEAH WELL I GUESS AS SOON AS WE HAVE THE RAW TEXT WE CAN PROBABLY JUST START WITH THE JAVA HASH MAP AND LIKE JUST HASH MAP OVER IT AND SEE HOW FAR WE GET I MEAN WE CAN PROBABLY ON A MACHINE WITH A FEW HUNDRED MEGABYTE RAM YOU CAN GO QUITE FAR YOU CAN WRITE IT ON BEEFY SO EVEN IF IT GOES WRONG AND EVEN IF IT HAS A MILLION WORDS BE 'CAUSE I HATE WORKING ON DICE IT'S AWFUL LIKE SO I CAN USE MY HOME MACHINE OH YEAH BURNING IT ON A LIKE WE SHOULD BE ABLE TO BURN THE WHOLE CORPUS JUST THE X. HMM AH I SEE I ASKED SUPPORT ABOUT THAT TWO DAYS AGO IN THE INFORMATICS BUILDING THERE OH SORRY IN IN APPLETON TOWER FIVE THE ONES CLOSEST T TWO MACHINES CLOSEST TO THE SUPPORT OFFICE SO I PRESUME OH WAIT I HAVE THE EXACT EMAIL I THINK HE'S TALKING ABOUT SORT OF THE ONES THAT THE RIGHT HAND CORNER FAR RIGHT YEAH UM THE THING IS LIKE YOU CAN ONLY BURN FROM THE LOCAL FILE SYSTEM SO IF IT'S FROM S WELL ACTUALLY I THINK IF IT'S MOUNTED YOU CAN DIRECTLY BURN FROM THERE BUT THE PROBLEM IS I HAVE MY DATA ON BEEFY AND SO I HAVE TO GET IT INTO THE LOCAL TEMP DIRECTORY AND BURN IT FROM THERE BUT YOU CAN BURN IT FROM THERE HOW BIG IS IT WITHOUT UM UH WE LOOKED THAT UP AND I FOR WE LOOKED THAT UP AND I FORGOT 'CAUSE I COULD JUST SAY AT UM GOING OVER S. C. P. ONE NIGHT AND JUST LEAVE IT GOING ALL NIGHT IF I HAD TO YEAH YEAH NO YOU YOU WE SHOULD BE ABLE TO GET IT AT I DON'T THINK IT WAS I DON'T THINK IT WAS A GIGABYTE IT'S YEAH I MEAN THE WAVE DATA ARE OBVIOUSLY NOT GONNA GET OFF THERE COMPLETELY BUT YOU KNOW WHAT I FIGURED OUT I'M QUICKER DOWN LOADING OVER BROAD BAND INTO MY COMPUTER THAN USING THIS HARD DISK THERE'S SOMETHING STRANGE ABOUT THE WAY HOW THEY ACCESS THE HARD DISK HOW THEY MOUNT IT WHICH IS UNFORTUNATE I'LL SEE IF I CAN S. C. P. IT I SUPPOSE HMM WHAT OPERATING SYSTEM DO YOU HAVE I'VE GOT A LINUX BOX AND A WINDOWS BOX SO BROAD BAND YEAH SO IF ANYONE OF US GETS IT WE CAN THEN JUST USE AN EXT HMM YEAH BURN IT TO C. D. OR YEAH PUT IT ON ON HARD DISK WHATEVER QUESTION IS IF YOU'RE NOT QUICKER IF YOU UH BECAUSE YOU SHOULD GET MASSIVE COMPRESSION OUT OF THAT LIKE FIFTY PERCENT OR SOMETHING WITH A GOOD ALGORITHM SO IF YOU COULD COMPRESS IT AND JUST PUT IT INTO A TEMP DIRECTORY LIKE IS HOW MUCH DO WE GET REALLY OKAY I DO LIKE I MEAN THERE'S NOT GUARANTEE THAT ANYTHING STAYS THERE BUT OVERNIGHT IT'LL STAY AND I THINK THE TEMPS USUALLY HAVE AH YEAH BUT THAT WOULD HAVE TO BE THE TEMP DIRECTORY OFF THE MACHINE YOU CAN S. S. H. INTO DIRECTORY OF S. S. H. YEAH BUT I CAN DO IT FROM THAT SESSION CAN'T I YOU CAN COMPRESS IT FROM A REMOTE SESSION AND S. C. P. IT FROM THE SAME SESSION YEAH THEY WOU THEY'D THEY'D PROBABLY HATE YOU FOR DOING IT BUT THEY'D PROBABLY THEY'D LIKE YOU MORE IF YOU S. S. H. UH INTO ANOTHER COMPUTER COMPRESS IT THERE AND THEN SORT OF COPY IT INTO THE INTO THE GATEWAY MACHINE DO YOU THINK THEY HAVE UM IF YOU S. S. HEY YOU KNOW IF YOU IF YOU S. S. H. AND THEY HAVE THIS BIG WARNING ABOUT DOING NOTHING AT ALL IN THE GATEWAY MACHINE YEAH TO YOUR HOME MACHINE YEAH I MEAN IT HAS TO GO THROUGH THE GATEWAY BUT CAN YOU NOT DO THAT IT'S NOT IT'S NOT EASY DEFINITELY THAT'S WHY I END UP SORT OF COPYING STUFF INTO THE TEMP DIRECTORY AT THE GATEWAY MACHINE SORRY IF THIS IS BORING EVERYBODY ELSE THIS IS JUST DETAILS AND HOW TO GET STUFF HOME FROM WHAT WE CAN PROBABLY JUST LOOK AT THAT TOGETHER WHEN WE'RE MEETING 'KAY UM I JUST UM WONDERED SO WHO'S UH THEN DOING UM THE FREQUENCIES ON ON THE WORDS BECAUSE I'M I THINK I WILL ALSO UM I COULD ALSO MAKE USE OF IT UM FOR THE AGREEMENT AND DISAGREEMENT THING BECAUSE I UM I IN MY OUTLINE I TALKED ABOUT UM USING THE UM DISCOURSE ACTS FIRST AND UM THEN IN THE CHUNKS OF TEXT I FOUND LOOKING FOR WORD PATTERNS AND SO ON SO UM I WOULD FOR EXAMPLE NEED THE UM MOST FREQ UM FREQUENT WORDS SO IF YOU CUT OFF ALL THAT I'D WON'T BE USE OR I CAN PROBABLY JUST IMPLEMENT LIKE A FIVE LINE JAVA HASH TABLE FREQUENCY DICTIONARY BUILDER AND SEE I WOULD YOU KNOW YEAH BUT I'D UH I WOULD LIKE TO LOOK AT THE FREQUENCY OF WORDS IN MY UM IN THE REGIONS OF TEXT I FOUND OUT TO BE INTERESTING SO I WOULDN'T NEED IT IT IT WOULD HAVE TO BE RE CALCULATED ONLY FOR MY SEGMENTS BUT TH FIRST UH HOW BIG ARE THE CHUNKS HUH HOW BIG ARE THE CHUNKS YOU'RE LOOKING AT UH UH MM I THINK IT WOULD BE YOU KNOW L AS AS BIG AT AS THE HOT SPOT ANNOTATION THINGS THAT'S QUITE SMALL YEAH THAT'S SOME UTTERANCES HMM YOU JUST DO THAT ON LINE 'CAUSE THAT WON'T TAKE LONG TO BUILD A LITTLE DICTIONARY THAT BIG WILL IT I MEAN JUST USE THE SAME TOOL THAT WE USE YEAH SO I WOULD PROBABLY JUST CONCATENATE ALL MY UM TEXT CHUNKS AND THEN LET'S SAY M I WILL RUN OVER IT OH YOU DON'T WANNA HAVE DIFFERENT COUNTS FOR EACH CHUNK BUT JUST LIKE SORT OF FOR FOR SOMETHING FROM OLD CHUNKS OH YEAH NO THAT'S YEAH SO ONCE I WRITE AN AR LIKE W IF I WRITE LIKE AN ALGORITHM WHICH DOES A HASH UM TABLE DICTIONARY WITH FREQUENCY FROM A RAW TEXT THEN THE RAW TEXT CAN BE ANYTHING SO HOW FAR ARE WE G UH HOW F HOW FAR ARE YOU GETTING RAW TEXT OUT OF IT DO YOU THINK UH I CAN GET ALL THE RAW TEXT BUT IT HAS TO BE ORDERED STILL SO OKAY WELL THAT'S GOOD BECAUSE FOR THE DICTIONARY THE ORDER DOESN'T MAKE A DIFFERENCE DOES IT SO YEAH SO UM I'LL GET THAT FROM YOU AND I'LL WRITE THE HASH TABLE WHICH GOES OVER THAT AND CREATES A DICTIONARY FILE SO FOR THE DICTIONARY IS IT OKAY IF I DO WHATEVER WORD BLANK FREQUENCY OR SOMETHING