I'VE GOT SOME OTHER STUFF THAT NEEDS DONE ON MATLAB SO IF YOU'RE NOT THERE AT TWELVE I CAN JUST WORK ON THAT SO YEAH I GUESS AT THE MOMENT NOBODY CRITICALLY DEPENDS ON LIKE THE NITE X. M. L. STUFF WORKING RIGHT NOW RIGHT LIKE AT THE MOMENT YOU CAN ALL DO YOUR STUFF AND I CAN DO MY L. S. A. STUFF AND I CAN EVEN DO THE DISPLAY TO A VAST DEGREE WITHOUT ACTUALLY HAVING THEIR SUPPLYING FRAMEWORK WORKING SO IT'S NOT THAT CRUCIAL YEAH I THINK SO YEAH ACTUALLY I NEED THE RAW TEXT AS WELL YEAH BUT I WAS I WAS I WAS MORE THINKING OF THE SORT OF THE THE WHOLE BROWSER FRAMEWORK AS A RUNNING PROGRAMME NOW YEAH I THINK WE ALL NEED THE RAW TEXT IN DIFFERENT IN DIFFERENT FLAVOURS DON'T WE YEAH YEAH JASMINE I THOUGHT YOU JUST SAID THAT YOU'D UH LOOKED AT EXTRACTING THE TEXT YES I DID BUT UM I'VE ONLY JUST GOT THE NOTES I HAVE TO STILL HAVE UH TO ORDER EVERYTHING BY THE TIME AND YEAH I THINK IT'S QUITE EASY AFTER THE YEAH YEAH YEAH SOUNDS PRETTY GOOD SO UM 'CAUSE YEAH I WAS HAVING A LOOK IN IT A LOOK AT IT AS WELL AND I NOTICED THE UM THE SPEAKERS ARE ALL IN THAT SEPARATE FILE SO DID DID YOU HAVE TO COMBINE THEM ALL AND AND THEN RE ORDER THEM YEAH B I UH W THAT'S WHAT I WAS UH THOUGHT THAT YOU JUST COMBINE THEM AND THEN ORDER THE TIME STAMPS ACCORDINGLY YEAH SO THAT'S APPROACH UM WELL I WAS GOING TO DO SO YEAH WE MAY AS WELL COLLABORATE OKAY UM WHAT I FOUND OUT WAS THAT THERE ARE QUITE A LOT OF THINGS WITHOUT WITHOUT S TIME STAMPS IN THE BEGINNING IN THE WORD FILES YEAH AND UH X. M. L. FILES YEAH THAT'S JUST AN I. D. OR SOMETHING I DON'T KNOW JUST NUMBERS YES BUT WHAT ARE THE OTHER THINGS THAT'S UH SOME KIND OF NUMBER F MAYBE THE FILE NUMBER OR SOMETHING THAT IS IN THE BEGINNING WHAT IS THAT I'M NOT SURE I WHAT YOU MEAN UM I THINK THERE ARE QUITE A LOT OF NUMBERS IN THE BEGINNING WHERE N THERE IS NO TIME STAMP FOR THE NUMBERS IT'S THINK THEY SAY UM QUITE A LOT OF NUMBERS AND BEFORE THAT UH UM THERE'S THIS NUMBER WAS IT YEAH THERE I ARE NUMBERS IN THE UM THE W. TAG BUT THERE ARE NO TIME STAMPS ARE THEY SPOKEN NUMBERS LIKE DO THEY LOOK LIKE THEY'RE UTTERANCES NUMBERS THERE'S THE NUMBER TASK ISN'T THERE THAT'S PART OF THE WHOLE THING HMM THAT'S AT THE END I THINK HER TIME OH RIGHT AT LEAST I SAW SOME HMM YEAH IT'S PROBABLY GONNA SCREW UP A LOT OF OUR DATA OTHERWISE IF NOT SURE IF IT WHAT IT DOES TO DOCUMENT IT WOULD PROBABLY MAKE THE YEAH IF IF YOU HAVE SEGMENTS FOR THAT PROBABLY THE SO THAT SHOULD BE REALLY EASY TO CUT OUT IT WOULD PROBABLY FIND ANOTHER NUMBER TASK QUITE EASILY SEEING THAT IT'S A CONSTRAINED VOCABULARY WITH A HIGH CO OCCURRENCE OF THE SAME NINE WORDS SO THAT WOU TEN WORD HMM YEAH SO I THINK IT'S ALSO SOMETHING THAT THEY THEY SAID THE NUMBERS IN ORDER RIGHT MM THEY HAVE TO READ NUMBERS FROM BECAUSE BASICALLY IT'S LIKE THEY'RE READING THEM AT DIFFERENT SPEEDS BUT YOU KNOW IN WHICH ORDER THEY ARE SAID ANYWAY ICSI HAS SOME REASONS FOR DOING IT THEY MUST HAVE BEEN PISSED OFF SAYING LIKE NUMBERS AT THE END OF EVERY MEETING AND ALSO UM THERE ARE DIFFERENT UM COMBINATIONS OF LETTERS B. R. E. AND SOMETHING LIKE THAT IS IT EVERYTHING ORDERED ARE THE TIME STAMPS GLOBAL OR UH ARE THEY LOCAL AT ANY POINT MM I THOUGHT THEY WERE LOCAL TO TH A PARTICULAR MEETING OKAY UH YOU NEED SOME DICTIONARY FOR THAT AT SOME POINT THOUGH LIKE YOU NEED TO HAVE SOME REPRESENTATION OF A WORD AS NOT NOT THAT SPECIFIC OCCURRENCE OF THAT WORD TOKEN BUT OF OF OF A GIVEN WORD FORM BECAUSE YOU'RE MAKING COUNTS FOR WORD FORMS RIGHT YEAH SO WE SHOULD WORK TOGETHER ON THAT BECAUSE I NEED A DICTIONARY AS WELL OH MINE'S JUST GONNA USE THE UM HASH MAP ONE IN UM JAVA 'CAUSE I'M ONLY GONNA DO IT ON SMALL DOCUMENTS IT'S JUST LIKE BEF UNTIL THE INFORMATION DENSITY IS UP AND RUNNING JUST SOMETHING TO GET GIVE ME SOMETHING TO WORK WITH OKAY DIDN'T YOU SAY THAT THE O THE ORD YEAH BUT FOR I'M JUST WONDERING FOR THE WHOLE THING DOES SOMEBODY WO WHO WAS IT OF YOU TWO WHO SAID THAT UM THERE'S SOME PROGRAMME WHICH SPITS OUT A DICTIONARY PROBABLY WITH FREQUENCIES IT'S UM I THINK IT'S JUST THE DICTIONARY IN THE FIRST PLACE BUT OKAY IS ANYONE OF YOU FOR THE FOR THE DOCUMENT FREQUENCY OVER TOTAL FREQUENCY YOU GONNA HAVE TOTAL FREQUENCIES OF WORDS THEN WITH THAT RIGHT LIKE OVER THE WHOLE CORPUS SORT OF OR WHY DOES IT NEED TO BE CLASSIFIED INTO LIKE DIFFERENT SEGMENTS IT'S BECAUSE UM RAINBOW IS A TEXT CLASSIFICATION SYSTEM AND I THINK IT'S NOT POSSIBLE TO HAVE JUST ONE CLASS THAT'S THE PROBLEM CAN WE JUST FILL A SECOND CLASS WITH JUNK THAT WE DON'T CARE ABOUT MAYBE WE COULD LIKE I DON'T KNOW COPIES OF SHAKESPEARE OR SOMETHING 'CAUSE IF WHAT WE'RE LOOKING FOR IS THE UM FREQUENCY STATISTICS I DON'T SEE HOW THAT WOULD BE CHANGED BY THE CLASSIFICATION IF WE NEED JUST FREQUENCIES MAYBE WE SHOULD JUST CALCULATE THEM BY USING PERL OR SOMETHING I DON'T KNOW WELL THERE MAYBE ANOTHER TOOL AVAILABLE YEAH IT'S QUITE EASY TO JUST COUNT AND S OR SORT THEM BY UM FREQUENCY W USING WHICH TOOL ARE YOU TALKING ABOUT BE CAREFUL WITH THAT LIKE MY EXPERIENCE WITH THE BRITISH NATIONAL CORPUS WAS THAT THERE'S FAR MORE WORD TYPES THAN YOU EVER THINK BECAUSE ANYTHING THAT'S SORT OF UNUSUAL GENERALLY IS A NEW WORD TYPE LIKE ANY TYPO OR ANY STRANGE THING WHERE THEY PUT TWO WORDS TOGETHER AND ALSO ANY NUMBER AS A WORD TYPE OF ITS OWN SO YOU CAN EASILY END UP WITH HUNDRED THOUSANDS OF WORDS WHEN YOU DIDN'T EXPECT THEM SO GENERALLY DICTIONARIES CAN GROW BIGGER THEN YOU THINK THEY DO I DON'T KNOW HOW YOU HOW MANY TERMS YOU CAN HANDLE IN PERL WELL YOU CAN PROBABLY ALSO YOU CAN PROBABLY PRE FILTER LIKE WITH REGULAR EXPRESSIONS EVEN JUST SAY IF IT CONSISTS OF ONLY DIG DIGITS THEN SKIP IT OR EVEN IF IT CONSISTS ANY SPECIAL CHARACTERS THEN SKIP IT BECAUSE IT'S PROBABLY SOMETHING WITH A DOT IN BETWEEN WHICH IS USUALLY NOT SOMETHING YOU WANNA HAVE AND UM I CAN'T REMEMBER WHO'S GOT IT MIGHT BE WORDNET BUT ONE OF THESE BIG CORPUSES HAS A LIST OF STOP WORDS THAT YOU CAN DOWNLOAD AND THEY'RE JUST BASICALLY LISTS OF REALLY UNINTERESTING BORING WORDS THAT WE COULD FILTER OUT BEFORE WE DO THAT IT'S LIKE THAT'S ONE THE PAPERS I READ