Word Vectors in the Eighteenth Century, Episode 1: Concepts

0. Introductions

The Prelude

This is the first in a series of posts about word vectors and eighteenth-century literature. I just started playing with word vectors recently, but I'm already deeply fascinated by them-as a method, as a concept, even as a style of conceptualization. So far I've made a word embedding model of ECCO-TCP and have been exploring some ways that vectors can raise interesting questions for 18C studies-such as, for example, model the interactions between well-known 18C cultural-semantic debates ("Ancients" vs. "Moderns", "Simplicity" vs. "Refinement", "Reason" vs. "Passion", "Invention" vs. "Judgment", etc).

But rather than keep on randomly exploring this new cyberspace for dozens of hours while never really writing anything down, and then kinda sorta forgetting exactly what I did months later when I try to write it up as a talk-what I usually do-these days, I have the urge to actually write down what I see, to draw it out on the spot, like Darwin, or Ted Underwood, would. :) So, I thought that I would try, again, hand to my heart, to reform my hermit-like, non-blogging ways. A weird, digitized-Wordsworthian breeze comes over me: and, with the inspiring examples of DH bloggers everywhere before me, I set out!

The Plan

I begin by pointing to why vector space semantics has become so interesting recently: the ability of new models to represent and predict, mathematically, semantic relationships between words as complex as analogy: Man is to Woman as King is to what?, you ask; Queen, the model replies. The next section re-approaches this same topic in an eighteenth-century context, close-reading word vectors by close-reading the analogy, Riches are to Virtue as Learning is to Genius, lying at the heart of Edward Young's influential arguments, in 1759, for original composition over neo-classical imitation. In the last section, I indulge sublunary thoughts on how modeling conceptual relationships through mathematical operations raises interesting questions about the status of concepts themselves, both those of the eighteenth-century and our own.

In the next post, I'll go more into the methods and maths behind vector space semantics: a public self-struggling to explain it to myself. And then in the posts following that, I'd like to narrate some of my recent explorations distant-reading a strange and multidimensional eighteenth century.

Previous work

But, before I go any further I should note the excellent DH work already out there on word vectors. Ben Schmidt has an incredible post on word vectors and DH, "Vector Space Models for the Digital Humanities" (25 Oct 2015). He has also released an R package for computing word vectors and instructions on how to use it. I learned a lot about vector space semantics from Schmidt and would refer everyone to his more capable introduction.

I've also been very much inspired by Michael Gavin's post, "The Arithmetic of Concepts: a response to Peter de Bolla" (19 Sep 2015). Gavin beautifully articulates the conceptuality of word vectors and how they can be leveraged for a cultural-conceptual analysis. Douglas Duhaime's post, "Clustering Semantic Vectors with Python" (12 Sep 2015), demonstrates how to find semantic clusters within word vectors, which seems to me a novel approach to generating semantic fields. And Lynn Cherney has used word vectors to hilarious and revealing effect with an Oulipo-style deformation of Pride and Prejudice.

1. Why all the hype?

The word vectors I've been talking about are more accurately a particular kind of word vector, one inside a particular kind of model called a word embedding model. Even though this kind of model has a longer history, there's apparently been a craze about this kind of vector space semantics for only the last few years-begun, in part, by Google, when researchers there unveiled newly efficient algorithms in 2013 for computing this kind of model, packaged in software they released called word2vec.

The hype, however, really arises from the shocking complexity and accuracy of the semantic relationships between words that word vectors within word embedding models are able to express. The canonical example is analogy. Word vectors are able to express and predict, mathematically, that "man" is to "woman" as "king" is to "queen"; or that "London" is to "England" as "Edinburgh" is to "Scotland". These analogical relationships can be expressed mathematically in terms of word vectors: V(woman) - V(man) + V(king) ≈ V(queen). Start with the vector for "woman"; then subtract from it the vector for "man", leaving behind only what is unique about V(woman) as distinct from V(man); then, add this distinct difference to V(king). You end up with a brand new vector position: V(woman-man+king). Which word vector, out of thousands of other words, is closest to V(woman-man+king)? In most word embedding models: V(queen).

Although these are toy examples, the possibilities for DH (as well as 18C studies) of an expressive model of, say, 18C semantic relationships, remain vast, and have only just begun to be explored. These possibilities seem especially fruitful for a conceptual analysis of culture-outlined, more recently, by Peter de Bolla in Architecture of Concepts and Michael Gavin's vector-based essay in response; but of course also by the long tradition of intellectual history, from Koselleck and Begriffsgeschichte, to Quentin Skinner and the Cambridge school, and to perhaps its most eminent practitioner in our own discipline, Raymond Williams.

2. A close reading of word vectors in the 18C

As a brief example of word vectors in action in the 18C, let's think back to the argument made by one of the most outspoken proponents of "original composition" over imitation of the ancients, Edward Young, in Conjectures on Original Composition: In a Letter to the Author of Sir Charles Grandison (1759). Young makes his ingenious argument by interweaving and correlating the literary-conceptual opposition between original and imitative composition with several other conceptual oppositions:

_Type of opposition Associated with original composition_ _Associated with imitative composition_
_Attributes of a Poet/Author_ Genius Learning
_Forms of social organization_ Organic growth Mechanistic commerce
_Forms of social value_ Virtue Riches