Yahoo demise: how can information be preserved? - Page 1
Forum section: electronics / general

author: tggzzz
author_rank: Super Contributor
timestamp: April 12, 2016, 08:21:12 am
content: It is, finally, beginning to look like Yahoo! will be broken up and sold in the forseeable future. When that happens it is reasonable to presume that bits will be turned off because they aren't profitable, with the loss of all information therein. Sometimes that can happen very quickly, e.g. the first large social media site in the UK, FriendsReunited.One or two Yahoo! groups contain valuable practical folklore and techniques, e.g. the TekScopes group. No doubt many other people will feel the same about their personal favourite groups.So, if worst comes to worst, what's the best way of (a) extracting and preserving, and (b) making accessible the information in such groups?

author: Rerouter
author_rank: Super Contributor
timestamp: April 12, 2016, 08:24:58 am
content: If they are free to view and download attachments, i would recommend a site archiving package, though i wouldnt be suprised if its not already in the many internet archives out there

author: German_EE
author_rank: Super Contributor
timestamp: April 12, 2016, 05:14:12 pm
content: Is it possible for the site administrators to download the message library to a zip file?

author: rx8pilot
author_rank: Super Contributor
timestamp: April 12, 2016, 05:23:44 pm
content: This is why I do not use cloud services for anything that I care to hold on to. All information that I care about - including email - I keep entirely on local drives.I have lost too much stuff when various companies change something, sell off, or go out of business.

author: MrSlack
author_rank: Frequent Contributor
timestamp: April 12, 2016, 05:24:00 pm
content: This is why the cloud is bad.Scraping software is probably the way to go. Then reprocess it into plain HTML, archive the attachments and files, zip it and stick it on a torrent site. I might do this myself as I have written a ton of scraping software over the years.

author: tggzzz
author_rank: Super Contributor
timestamp: April 12, 2016, 05:44:54 pm
content: Quote from: MrSlack on April 12, 2016, 05:24:00 pmMy view is that it is the triumphant re-invention of timesharing bureaux.Thise with long memories will recall the shouts of delight when PCs became widespread - because then the user owned and controlled their own data and processing, rather than having it under the control of white-coated magicians in freon-protected aircon halls.The Hindus are right: it isn't time's arrow, it is time's wheel.

author: SeanB
author_rank: Super Contributor
timestamp: April 12, 2016, 05:55:52 pm
content: Send a message to them to move here, they will likely get a new forum, and can only improve this one as well.

author: Kilrah
author_rank: Unknown
timestamp: April 12, 2016, 05:59:11 pm
content: I still have a 4GB mostly-working archive of an MSN group I cared about when those closed about 10 years ago, grabbed with whatever web crawler that actually wasn't very intelligent and du(tri,...)plicated lots of stuff.And yes, everything I can is stored on local hard drives. Would hate if Flickr went down though.

author: Cubdriver
author_rank: Unknown
timestamp: April 12, 2016, 06:09:21 pm
content: Quote from: SeanB on April 12, 2016, 05:55:52 pmThat was my first thought as well, though certainly not my place to do as this is Dave's house - it would be great to get all those HP and Tektronix guys on here. I'm on both groups (mostly as a lurker), but I much prefer a forum format like this to the klunky Yahoo mailing list, and have never had much success searching on their main pages - it's painful to do in my experience at least.-Pat

author: edavid
author_rank: Super Contributor
timestamp: April 12, 2016, 06:27:06 pm
content: 1. Yahoo itself does not provide any way to export or archive group content. Most of the Yahoo group downloading scripts out there stopped working when Yahoo ruined everything with their "Neo" transition.2. Many of the crotchety old members are adamant that they will never, ever use a forum instead of an email list.3. groups.io looks like a good option, since they will suck over the Yahoo message archive, but who knows how long it will be around. If I were a group owner, I would go with them if they would be willing to give me an offline archive.4. But, if we're talking about TekScopes and hp_agilent_equipment, neither group owner is exactly proactive, so it's hard to say if they will get it together to do something before it's too late.

author: MrSlack
author_rank: Frequent Contributor
timestamp: April 12, 2016, 06:49:59 pm
content: Quote from: tggzzz on April 12, 2016, 05:44:54 pmThis.Then again I work for a financial cloud company so I'm a giant hypocrite.

author: tggzzz
author_rank: Super Contributor
timestamp: April 12, 2016, 07:02:52 pm
content: Quote from: Cubdriver on April 12, 2016, 06:09:21 pmPersonally I like the email variant since it is trivial for me to personally archive whatever is interesting to me - and in a decentthreadedformat (cf the linear format onallweb forums).For newcomers and people with new interests, the Yahoo search facility ismuchmore effective than that on this forum.But most important is the ability to retain all the existing knowledge. Currently groups.io looks like a good starting point, but this is a good time to explore possibilities.

author: Kilrah
author_rank: Unknown
timestamp: April 12, 2016, 07:23:06 pm
content: Quote from: tggzzz on April 12, 2016, 07:02:52 pmTo me email lists are an archaic unuseable thing, and the threaded format is just a thing that's misused 90% of the time and only leads to a lot of mess because going through topics within a topic is just never-ending.Something about taste and points of views obviously makes a merge difficult...

author: SeanB
author_rank: Super Contributor
timestamp: April 12, 2016, 07:41:28 pm
content: I still have a list server running........ Most of the time the only posts are the daily 2-3 spam emails sent to it from random senders, though I get around 1 non spam per year from somebody looking for info. Only reason it is still up is that storage space is cheap, and there is another list server or ten running on the same virtualised server anyway. Might be 0.01% extra CPU time on the machine. As it is a moderated list, and I am the BOFH admin, it is very quiet.....

author: tggzzz
author_rank: Super Contributor
timestamp: April 13, 2016, 12:30:12 am
content: Quote from: Kilrah on April 12, 2016, 07:23:06 pmWhich, of course, says as much about you as about email lists.QuoteWhich indicates you don't understand.You can't "misuse" a threaded format. It justis. The threading is completely and solely defined on the basis of which email is a response to which other email. There is no concept of a "topic within a topic".The threading is implemented byyourmail user agent. You control and select it independently of anyone else, and can change it on a whim. For example, one of my email folders contains 8958 emails, going back to 2005. I flip between linear-date-order and threaded-date-order several times a day, and each flip takes a small fraction of a second.The same is true of usenet postings, where I also get my reader to highlight emails by anyone I'm particularly interested in, e.g. one of the authors of "The Art of Electronics".You cannot do that with any web-based forum.

author: DimitriP
author_rank: Super Contributor
timestamp: April 13, 2016, 01:30:05 am
content: Quote from: MrSlack on April 12, 2016, 05:24:00 pmNo need to re-invent the wheel...http://www.personalgroupware.com/

author: Wilksey
author_rank: Super Contributor
timestamp: April 13, 2016, 01:33:36 am
content: Cloud! Yay! Did no-one learn their lesson with Geocities?

author: tggzzz
author_rank: Super Contributor
timestamp: April 13, 2016, 01:41:58 am
content: Quote from: Wilksey on April 13, 2016, 01:33:36 amYes, they did learn.Then new people come along and repeat old mistakes. That's compounded by people presuming old=bad, new=better. (Ah! The arrogance of youth!)

author: station240
author_rank: Unknown
timestamp: April 13, 2016, 03:15:21 am
content: I'm seeing an even bigger problem emerge with chatrooms being used for technical support, of course it's a cloud system.There is no archive of anything, so unlike the forum, the technical knowledge is simply lost hours after it's created.

author: MrSlack
author_rank: Frequent Contributor
timestamp: April 13, 2016, 05:43:48 am
content: The joyous chat room situation was solved by our glorious support team by saving all chat transcripts, in jira, in the cloud, which we don't backup at all... Sigh...

author: hendorog
author_rank: Super Contributor
timestamp: April 13, 2016, 06:12:11 am
content: Google - ironically? - found this...https://sourceforge.net/projects/grabyahoogroup/and thishttp://www.usenix.org.uk/content/yahoo_mbox.html

author: German_EE
author_rank: Super Contributor
timestamp: April 13, 2016, 03:27:11 pm
content: From the web page of the first link:" The new neo interface has broken the script and I have not found any easy way to work with it. For now the script is not being actively maintained till a solution materializes to access the relevant data easily."It looks as if Yahoo have thrown a spanner in the works.

author: rrinker
author_rank: Super Contributor
timestamp: April 14, 2016, 12:28:09 am
content: That horrible "Neo" interface is a couple of years old now. Yahoo threw a spanner in the works alright, that was the beginning of the end. Prio to "Neo", Yahoo Groups was actually pretty useable. Wasn't difficult to search or find specific posts. Neo broke all that, and overnight it all became an unusable mess. I'm on more than a dozen, mostly model railroad related, though I only get email from about 5 of them. I rarely post because it's almost impossible. And not even WYSIWYG, in the editor box all you see is your text you've types, if you miss the little check box you end up quoting the full contents of EVERY post in the thread that came before your response. Horrible UI design.Then they bring in an absolute boob to run the company - she might be pretty to look at but she has the business acumen of a rock. All of their news sources have turned into pure clickbait and downright malware links, it's all those sort of sites where a half page story is spread across 10 pages so they can present even more ads. They used to aggregate all the popular comics, they tossed that. Fine, no ad revenue for you, I get my Dilbert direct from the source now, let Scott Adams get the revenue to hits on his site.

author: hendorog
author_rank: Super Contributor
timestamp: April 14, 2016, 12:53:10 am
content: Quote from: German_EE on April 13, 2016, 03:27:11 pmYeah I saw that too, I figured it was worth highlighting just in case someone was up for fixing it.Hrm, what about this:http://www.archiveteam.org/index.php?title=Yahoo!_Groupshttps://archive.org/download/yahoo_groups

author: XOIIO
author_rank: Super Contributor
timestamp: April 14, 2016, 08:15:23 am
content: Now we can only hope bing goes next.

author: SeanB
author_rank: Super Contributor
timestamp: April 14, 2016, 07:38:23 pm
content: Quote from: XOIIO on April 14, 2016, 08:15:23 amPlease, NO! You need a reference design for what is about the worst you can make a search engine before it only gives random results. Currently it is only pseudo random except where it is scraping a google result.

author: EEVblog
author_rank: Unknown
timestamp: April 15, 2016, 03:33:40 am
content: Quote from: tggzzz on April 12, 2016, 08:21:12 amDavid2 wrote a script to extract all the post from Yahoo Groups, it's on my github page:https://github.com/EEVblog/Yahoo-Group-Downloader

author: rx8pilot
author_rank: Super Contributor
timestamp: April 15, 2016, 04:39:13 am
content: For some period of time, this can be a small biz for someone willing to organize archiving for various Yahoo services that are not easily done by the layperson. Drop the archive on a cloud location and trade a couple of bucks. The customer can pull the archive and have control of their data.

author: Ian.M
author_rank: Super Contributor
timestamp: April 15, 2016, 04:46:37 am
content: I'm surprised noone's mentionedhttps://groups.yahoo.com/neo/groups/LTspice