I have begun to integrate indexes into the Distant Reader for the purposes of creating and downloading previously created data sets ("study carrels"). This posting introduces the work done to date.

The Reader creates and provides services against data sets afffectionately called "study carrels". To create a study carrel the student, researcher, or scholar assembles a set of files for analysis into a single directory. The directory is used as input to the Reader's build command, and the result is a data set that can be modeled for the purposes of addressing research questions. That said, you would be surprised how difficult it is for people to create a directory filed with files; now-a-days we seem distribute links to splash pages instead of the content itself. Sigh! The Reader's set of indexes is intended to make it easier to create a set of files for analysis and thus demonstrate the distant reading.

To date, a small set of indexes have been created. They include:

The idea behind the indexes is this:

Yes, at first glance, the process seems complicated, but a whole lot of it can be automated.

"Again, why are you doing this?" Because we continue to be drinking from the proverbial firehose; because we are suffering from information overload; because typical search results return so many relevant items, and we few means to really read and analyze all of them. The Distant Reader is intended to address this problem and more.

Finally, software is never done. If it were, then it would be called "hardware". That being the case, the links above will probably break sooner than I desire, but the process will remain the same: 1) search index, 2) refine result, 3) create data set, and ultimately, 4) do analysis -- read.