Thoughts on the fragility of digital systems

I, like some 17,000 other Elgg/Eduspaces users, got an email this weekend announcing the closing of the Eduspaces service. I'm sure that a fairly large number of them thought, also like me, "What's Eduspaces?"

Eduspaces, as it turns out, is/was the free host for the Elgg open source "social platform". I do remember signing up for an Elgg account a couple of years ago (a class, a workshop?), but I didn't remember the Eduspaces name. There are blogs, file storage, discussion areas, etc., in the Elgg system, and it will still be available for those who want to install it on their own servers. Just the hosted implementation at Eduspaces.net is going away.

Luckily for me, I had signed up and never used the account. I have no content to worry about in Eduspaces, but that certainly isn't true elsewhere. I have blogs, pictures, documents, video, audio, and combinations of things up all over the Internet. I have enough accounts with services that I've tested out that I've had to implement a system for keeping track of them. (I'm using the "private" setting on ClaimID.com to keep links and usernames/emails for services as I sign up for them. Of course, it doesn't do me any good for those things I've already forgotten about!)

Migrating content is something that everyone who uses any sort of digital system should be aware of. When you sign up for a content tool on the web, whether it be a photo storage site or a blog or a discussion group, you should ask if you can export your content in a reasonable manner (and printing pages and pages of content should not be considered a reasonable manner). When I started a project on Zoho Notebook recently, I was happy to discover that the content can be exported as a web page.

It's not just a Web 2.0 problem, though the fragility of the business models for many of these projects makes it more obvious. SCSU is about to lose access to the previous version of our WebCT system, so those with old courses need to migrate their content NOW! (Faculty: contact Stan Walonoski right away if there is old course material that you want back; students: contact your instructor.)

On a different note, I was recently having a discussion with someone who wrote a thesis quite a while ago (in computer eras, anyway). He doesn't have a print copy, and is having trouble reading the floppy disks. First problem--when was the last time you saw a computer with a 5 1/4 inch drive? Second problem--what version of Word Perfect is this? Third problem--are the files still good? (He's found someone for problem one, has some leads for problem two, and is now waiting for the results on problem 3.) I've got dozens of disks (3 1/2 inch, mostly) with files that I wanted to save back when, but most of them aren't readable now, either because I don't have the software or because the files have gotten corrupted over the years. Online storage would help the hardware problem (even 3 1/2 drives are getting scarce), but not the file format problem.

As we document our educations, our careers, and our lives increasingly online, we need to think about the fragility of digital systems. Do you backup? Do you save important "artifacts" in multiple formats? (A .txt file is a .txt file and probably will be for a very long time, thank goodness.) Do you check if your online accounts have mass export features? Do you keep track of what's where, or, like me, do you have bits and pieces scattered around from whatever was new and interesting at the time?

And please deposit a print copy of your thesis with your library.

Open Yale Courses

Our neighbor in New Haven, Yale University, has joined the amazing community of prestigious universities offering free access to course materials online. Yale's offerings, called Open Yale Courses, are in the form of video, audio, and accompanying materials like problem sets (with answers), exams, and lists of assigned readings: essentially the entire course except for interaction with the faculty member. Seven courses are currently available: Fundamentals of Physics, Frontiers and Controversies in Astrophysics, Modern Poetry, Death (Philosophy 176), Introduction to Political Philosophy, Introduction to Psychology, and Introduction to the Old Testament. At least 30 courses are planned for the next few years.

The video is high quality, filmed by university videographers, and tailored to the needs of the course. An article in Inside Higher Ed highlights the importance of displaying the board during a physics course, to show the discussed equations and examples. Transcripts and audio are available, and the video is available in several formats, including high bandwidth downloads.

All the materials are licensed via Creative Commons, so the materials can be reused noncommercially as long as credit is given. Yale has already entered into agreements with the governments of China and India to show the videos on state televisions, and individual faculty at universities in China, India, Mexico, Argentina, and Ethiopa have announced their intentions of including Yale materials in their courses (via the Yale Daily News).

Are you teaching or taking a similar course? These courses, as well as the other Open Courseware courses at other universities, represent a rich source of ideas and supplementary materials for instructors and students. These are wonderful for instructors teaching a new course or updating an old one. Or think of them as guest lecturers who are always available (just not for questions afterward). For students they represent the ability to get another perspective on a topic, and the access to world renowned scholars, without paying world renowned tuition.

And, of course, none of us (at least no one reading this) have an excuse anymore for not being lifelong learners.

Online education on NPR

National Public Radio broadcast the first of a two part series on online education this morning on Morning Edition. According to the story, one in five current college students is taking at least one online course. The story also includes interviews with online faculty, one of whom estimates that an online course takes twice as much work, in prep and teaching, as an on ground course. There is also a link to the Sloan Consortium 2007 survey, which is the source of the 1 in 5 statistic.

Look for the second part of this series tomorrow.

"How Google Books is changing academic research"

Jo Guldi is a Ph.D. candidate in California looking for information on roads in the 18th and 19th centuries. She had, in proper library research fashion, explored the contents of the Yale, Harvard, and British Libraries, California libraries, and ultimately the contents of a large number of North American libraries via interlibrary loan.

Then she stumbled upon Google Books. And turned up 20 new sources related to her research in a single search. That's 20 new sources, sources which she had no idea existed from her library research.

How did this happen? I see two major contributions, one fairly trivial, one major. The trivial one was her search. According to her blog post, she did a search for "roads", a single keyword across the vast collection of Google Books. While she doesn't say so, you can easily limit your search to fully available sources, which generally means out of copyright--mostly pre-1920 or so. This limits the search to works that are most likely to be of use to her, 19th century texts, since those are both out of copyright and in good enough shape to be scanned. In short, she was lucky, and lucky with a search that few librarians would have recommended in a non-subject specific index. (Confession: I actually do these sort of searches in general databases occasionally. When you are having trouble, sometimes it's worth doing single keyword searches to see if one particular keyword is limiting your search too much. At least you get a sense of the extent of the universe in which your more complex searches are operating.)

However, I don't think that is the real message of her search. Google Books also searches the full text of the books (both in and out of copyright). No library catalog can do that. And, as librarians know all too well, there is no way that a typical library catalog record can record every topic covered by a book. A library catalog is, in a way, a simulation of a library. It models the library in a way that makes it easier to manipulate, but the information available is degraded or simplified. (If a simulation has all the information contained in the system it is simulating, it is that system, not a model. Simulations work because non-essential information is stripped out. However, sometimes you are wrong in what is non-essential. Library catalogs work fairly well for many needs, but they are not suited for every single need.) Google Books increases the amount of information within the simulation, increases it to an amazing extent.

It is still a simulation of a library. Copyrighted books are not available fully. You lose some physical knowledge of the book. It doesn't contain the range of books in a full research library (though Google would like to, I'm sure.) Google has not replicated the metadata produced by the experts (librarians) who cataloged the books in the first place. However, for what it does contain, it's hard to argue that it's not a better simulation of a library than your average library catalog is. The major loss is that metadata, which Google attempts to replace with tables of contents, indexes, and a "Key words and phrases" section.

For those unfamiliar with library metadata, I'm mostly referring to subject headings. Subject headings, and other controlled vocabulary systems, allow the assignment of one or more subject terms to a book, article, etc., with the specific terms being consistent over the entirety of the system, and not dependent on whether or not the author used that term. It's related to the way libraries shelve books by subject, so that all the gardening books are together, or all the versions and criticisms of Hamlet end up on the same shelf.

Google's "Key words and phrases" imitates this, but is strictly dependent on the words used in the actual book. I'm assuming that this is an automated process, so this probably related to the frequency of use, and the placement of the words. I would guess that a word used over and over again will score high for inclusion on this list, as will words used in the title, as chapter headings, etc. It should lead to some interesting choices: a book of General Sherman's letters includes "affectionately yours" as a key phrase. It's an interesting fact of letter conventions at the time, but not likely to be a good search phrase. (In case you are curious, the phrase "affectionately yours" turns up 1730 books in Google Book Search, mostly, though not entirely, books of letters. OK, I can see a use for this search, but how often to you need to know the relative frequency of different salutations in different eras? Getting back to the subject, we do loose the specificity of the LC subject heading "Correspondence"--used with a name to specify the letters of this person.)

I hope that Google will see the advantage of not tossing out decades of metadata that would only increase the usefulness of their system. Just add subject headings to the keywords already in use. It does lead to an interesting question of who owns library catalog records. Could Google buy them from someone? And from whom? LC? OCLC? Individual libraries as the books are scanned? Are they copyrighted, or is an individual record similar to an entry in a phone book?

To jump back to Jo for a final moment, she also points out one other service that Google Books performs: multiple access. She found a book that was invaluable to her research. She had kept it for months, paying overdue fines and all, so that she could ensure needed access to a book that she had had such a hard time getting a hold of. Here comes Google Books with instance access, as long as she needs it, and in no competition with any other researcher. (Previously, I had mentioned a tip for using Google Books for copyrighted books before putting in interlibrary loan requests.)

I'm not sure where Google Books is going, or even where academic research is going, but I do conclude that Google Books should be one of several avenues of library research that good scholars should perform.