I just put together a position statement for a particular venue, but realized in the process that I was basically writing the first draft of my dissertation proposal. So I thought I would post that part here.
Digital technologies increasingly shape the ways in which we know about and engage with the world. They mediate our perceptions, actions, and interactions with others. Their design and configuration determine what is possible, and shape the social interactions and culture that emerge around and through them.
Any time there is a technological device or infrastructure that captures activity and makes it visible so others can see it, there is the potential for a social space: a space of mutual awareness of presence and activities, and thus the possibility of interaction. Displayed records of activities might be designed to serve as social cues, but they can also do so unintentionally. For example, a shared network file system is not seen as social at all; yet even something as simple as the time stamp of when a document was created or modified and the name of a user can indicate that a person was at her computer and working on a particular project at that time. Even on a blog, that is explicitly social, there are cues that might not be seen as contributing to sociality but that nonetheless provide social information. Seeing the name of an author and a time stamp on a post sends the signal that this person was there at this particular time, which can lead to a sense of being in a shared space at the same time. Other mundane things like an instant messenger buddy list that gives a sense for who else is online (or not) provide information about what is going on in one’s world. And of course the amount and types of information that are captured, transmitted and displayed through technological means are only increasing.
As we design things, we are potentially not only setting up flows of information and social spaces, but also plugging them into and supplementing other flows of information and social spaces. Just as we can in the physical world sense something like a coming thunderstorm or the mood of a crowd by putting together a multitude of subtle cues in the environment, we can also get a sense for the world in a different way by putting together a multitude of mediated cues that bear witness to other types of activities.
There is currently no good language with which to make visible and critique the types of social spaces that are set up by the architecture of things that articulate these interactive spaces. We can talk about how values and interests are built into designs (as in STS) and about the culture that emerges around them (cultural studies). Yet neither of these areas of inquiry can speak directly to the design of actual artifacts, and point to the specific places where the technical rubs up against the social.
While this type of analysis is important for pedestrian activities and interactions, it becomes even more crucial when it comes to pervasive computing elements such as embedded sensors and ambient displays. We understand how we can perceive and be perceived in the physical world through touch, sight, hearing and smell. But there are now new ways of detecting and making activity visible, ways of being in and knowing the world around us that have been made possible only in recent years.
We need ways to think about things as they relate to whole systems and the embodied experience of them in order to inform design and enable critique. The goal of my dissertation is to develop such a language and supporting theoretical framework: what I am calling the architecture of interaction.
Taking a cue from architects, who are used to thinking about how concrete designs shape experience and interaction, I argue that digital technologies can perform a similar function. Digital architecture provides an infrastructure that can shape flows and patterns of activity; provide or restrict access and visibility; articulate social spaces and connections between them; shape the mood of spaces through aesthetic and historical properties; and shape subjectivities in relation to the environment.
A key part of where architecture (both physical and digital) meets the social is in its tangible substrate. A substrate can, among other things, allow or prevent action and determine what is visible. For example, a concrete wall prevents one from passing through it and also from seeing through it, while a glass window allows one to see but not pass through it. In the digital realm, the architecture of a blog can similarly determine what is possible (e.g., posting, commenting, sharing through social media) and visible (e.g., counter of how many ‘retweets’ or ‘Diggs’ a post has received that records and makes that activity visible).
Another key aspect of architectural substrates is how they can come to record and bear witness to activities that happen in, on and around them. The nature of the substrate determines the types of traces that can be left behind. So for example, grass can have a path worn through it by many people walking over time. The substrate of the grass can record the activity of walking in the trace of the path. Air can also be a substrate, as smells of things like food, smoke or perfume can linger even after their original sources are gone, suggesting that they have been recently present. Something like a concrete wall may be sprayed with graffiti that bears witness to the presence of hooligan artists. And deep ruts in the pavement at an interstate exit by a truck stop are a record of the many trucks that have sat there at the traffic light.
In the digital realm the relation between substrate and trace becomes more complicated, as it can be largely determined by the designer and may be non-obvious to users. Indeed, this is one of the particular challenges of pervasive sensors, as the particular activities they record, and the ways in which they are later displayed, may not be readily apparent. A more straightforward example is that of the blog post mentioned earlier that has a tool for sharing the post via social media, and a number associated with each that also serves as a counter for the number of times other visitors to the blog have utilized it. This particular element of the blog serves as a substrate that captures the activity of sharing via social media, resulting in the trace of the number in the associated counter. And of course recording and sharing of activities is a fundamental part of the architecture of the widely used social networking site Facebook, which is in and of itself a complex hornet’s nest of issues involving activities, traces, visibility, and sociality.
When it comes to digital architecture, there is clearly a tremendous flexibility in the relation between activity, trace, and perception. Ambient sensors and displays in particular are amenable to a wide range of captured activities and displays in which the relation between them is not at all clear. But in addition to the potential confusion, there is a much more serious issue at stake regarding mediated perception of reality.
When seeing a desire path that has been worn through grass, there would be little reason to believe that it has been (or even could be) caused by anything other than a number of people walking the same trajectory over a period of time. But technological traces and the perceptions they engender can be much more easily subverted. For example, a lawyer may have the sending of an email delayed until the early hours of the morning to give his client the impression that he is up late working on the case, counting on the fact that the client will trust the substrate of the email system to objectively capture the email time stamp as a trace of when the message was sent. Or consider the classic stunt of the ingenious crime show villains who replace live surveillance camera feeds with a pre-recorded video loop. In this case the hapless security personnel are duped by what they naturally consider to be a straightforward relation between camera, reality and displayed video, but which is actually a lie that plays on assumptions of how the technological apparatus normally functions.
It is thus important to understand the nature of traces, both in order to make the operation of technologies transparent to users appropriately and to critique the relation between perception and reality that they support (or subvert). I refer to this issue of interpretation as the hermeneutics of technologically-inscribed traces. This is a key part of the architecture of interaction, since traces are the single means of making actions visible and thus entirely mediate perception (unlike in the physical world where several senses can be used).
Heather Wiltse, PhD 