cognitive engineering

ldclark February 25, 2020

Designing the Human-Technology Relationship, Uncategorized Brain-machine interface, cognitive engineering, display design

Cognitive Symbiosis: A Student’s Take on Connecting Human and Machine Intelligence

In the second year of my undergrad degree at UCF, I took a course that fundamentally changed my perspective on the relationship between humans and our technology. The class was called “Ethics, Society, and Technology: Preparing for our Cyborg Future”, and it was co-taught by a philosophy professor and a cognitive scientist. Over the course of the semester, we explored how humankind’s technological tools have evolved over time, how those changes have shaped our societies, and how future innovations might change things even more down the road. It was an epic blend of emerging tech, science fiction, and philosophy, and I looked forward to it every week.

A little more than a month into the class, we were asked to write a paper in response to some arguments raised by the author Andy Clark (unfortunately, no relation) in his book Natural Born Cyborgs. Clark had argued for the value of viewing humans and the tools we use to enhance our cognitive abilities as a single unified system, and he put forward the idea that human beings are already “cyborgs” in the sense that we regularly integrate external tools into our physical and cognitive capabilities. I’ve since learned that this idea is very similar to Dave Woods’ concept of a Joint Cognitive System— a foundational concept in my chosen field of cognitive systems engineering.

I had a blast with this assignment, spending hours identifying, testing, and refining my first set of positions on the functional dynamics of the human-machine system. Three years later, I look back on the end result and see some early hints of the ideas that still drive my fascination with designing machines that can make us smarter. Now that I have a website (and the unilateral power to share anything I want with whoever happens to stop by), I thought it might be cool to pass along this little piece of my professional origin story in a more reader-friendly format. I hope you enjoy!

Case Study A Morning at the Coffee Shop

The year is 1921, and as I step out of the brisk winter air into a bustling cafe I feel a rush of warmth as the smell of fresh coffee grounds reaches my nose. As I make my way toward the counter, a young woman, clearly disillusioned with the poor fellow sitting across from her at the table, stands up in the midst of a half hearted explanation and sheepishly eyes the door, nudging her chair into my path in the process. I shoot the lad an empathetic glance as I quickly shuffle through the narrow space between her chair and the adjoining table and take my place in line at the counter.

As I survey the coffee shop and chuckle as I watch the young woman frantically attempting to hail a taxi outside the front window, I remember a particularly horrid date I’d experienced at that same cafe some years before. The extent of my luck that morning was receiving a free danish when the barista spilled coffee on my daily copy of the New York Times, and what a danish it was! I must say I’d rather fancy one now, if only I could remember what it was called. I reach for a menu and scan through the offerings, and the description of a particular pastry catches my eye. As I read it, details of that morning come flooding back to mind. “This must be the one”, I conclude, “I’m sure of it”.

Before I know it I’m stepping away from the counter, coffee and danish in hand, making my way toward my usual table in the corner by the front window. Out of the corner of my eye I spot a familiar face, and briefly divert my course. “Excuse me sir,” I interject, leaning over carefully so as not to spill my coffee, “I believe I may know you from somewhere.” The man is similarly puzzled with the familiarity of my face, but following a few simple queries as to where we might have met I realize that I am talking to none other than Dan, by old childhood friend from Dallas. He invites me to have a seat at his table, and as we take turns recalling our exploits of years past a deluge of long forgotten memories come flooding back to me in vivid detail. An hour later, I bid goodbye to Dan and step back out into the chilly winter morning, realizing all at once that I might now be late for class.

Natural Born Cyborgs The Notion of Human-Machine Symbiosis

Knowledge is the centerpiece of the human cognitive machine. It guides thought and action, both necessitating and directing each successive iteration of the complex interplay between the cognitive and physical operations of the human person. In his book Natural Born Cyborgs, philosopher Andy Clark (2003) describes the tendency to incorporate external resources into the cognitive system as a defining characteristic of humanity and a central feature of our unique capabilities. “We are already primed by nature to dovetail our minds to our worlds.” he argues, “Once the world starts dovetailing back in earnest, the last few seams must burst, and we will stand revealed: cyborgs without surgery, symbionts without sutures.”

In Clark’s human/machine symbiosis, the human element of the system is constantly receiving information from both the biological brain and the technological world. Using the example of a man being asked for the time, and pointing out that he is apt to respond “Yes” when asked “Do you know the time?” even when he must consult his watch to retrieve that information, Clark argues that “the line between that which is easily and readily accessible and that which should be counted as part of the knowledge base of an active intelligent system is slim and unstable indeed”.

To Clark, information that is ready at hand should be considered part of the knowledge base of the individual, regardless of whether it originates in the biological brain or the technological world. However, Clark notes, there is a tendency to view knowledge in the brain as somehow more valuable or complete than knowledge stored in the environment. This tendency, he posits, arises from a commonly held illusion of the brain as “the sole and essentially insulated engine of mind and reason”. This conception, he argues, is effectively a physicalist extension of Cartesian dualism—“the old idea of spirit-stuff in modern dress”.

While the cognitive distinction between information from the brain and information from the environment does to a certain extent constitute an illusion, like many illusions I would argue that it likely has an underlying evolutionarily adaptive purpose. Just as the size-weight illusion provides a functional measure of the relative ease with which objects can be thrown, the common bias toward biologically held knowledge could well have been a function of the relative usability of information from biologically and ecologically based sources. In this context, the tendency to view the biological brain as a privileged operator among a host of slave systems could be seen as an artifact of the past and present state of the human-technology relationship. As knowledge stored in silicone becomes just as readily accessible as knowledge stored in brain tissue, this distinction may lose its functional relevance.

Transactional Memory Storing Information in the World

My fictional experience in the 1920’s coffee shop provides a strong example of this waning body-environment cognitive dualism. When I walk into the cafe and see the woman standing up from the table, I retrieve information from my biological memory about situations similar to the one playing out before me, and this knowledge directs me to respond as I do (with an empathetic glance). As I approach the gap between the young woman’s chair and the adjoining table, knowledge of my body’s dimensions and physical capabilities, acquired through years of action and sensory feedback, informs me that I can turn sideways and shuffle through the narrow space.

In both of these cases, knowledge stored in biological memory guides and informs my actions, and few would dispute the claim that I “know” how to respond to these situations. In contrast, when I decide to purchase the danish and find that my biological memory lacks the knowledge I require, I retrieve that information from a location in the environment where I know it to be stored (on the menu), and this “knowledge in the world” assists me in reaching a conclusion. In my conversation with Dan, I enter into a complex cognitive interplay in which his memories of our childhood experiences help me to call to mind my own memory for those experiences, and our collaborative elaboration enhances our collective memory of the past. In both cases I am engaging in transactive memory, actively retrieving information that I know to be stored in the environment. To a body-environment cognitive dualist, I do not actually know the name of the pastry or the name of my former next door neighbor in Dallas—I merely know where to find them.

Blurring the Line Could Biological Memory Lose Its Edge?

In the world of the 1920’s coffee shop, biologically held memory was functionally advantageous insofar as it was more consistently present, immediately accessible, and effective at anticipating the goals and needs of the individual than other available forms of memory. For example, while a paper menu can present a particular set of information to a literate diner, it cannot anticipate his desires or recall precisely how rare he likes his steak. Similarly, information stored in another person is only accessible when that person is present, and as such one could not realistically rely on this form of transactive memory as a consistent knowledge source. In this way, knowledge stored in the brain transcended the functional constraints imposed on the available media of ecologically available information, providing the individual with a sense of mastery and ownership of the information that would not be there were that knowledge stored in the environment.

At its core, then, it is possible that the cognitive advantage of “knowledge in the head” over “knowledge in the world” may have arisen from the lack of a reliable, high-bandwidth connection between the biological brain and ecologically stored knowledge. As such, in a future of transparent, ubiquitous technology seamlessly integrated into the human cognitive system, the storage of information in the biological brain may lose its functional advantage. If a non-biological memory agent were as consistently present, readily available, and situationally aware in its presentation of information as biological memory, it could effectively meet and potentially exceed the memory functionality of the biological brain.

As the capabilities of technological systems become more fluidly integrated with the processes of perception, cognition, and action, a new host of cognitive and physical operations become available, and these new possibilities open the door to the expansion of the individual’s cognitive system beyond the immediate physical location of his or her body. I suspect that such an expansion will further blur the already thin line between our biological and technological cognitive subsystems, as the primacy of knowledge and action of the biological variety loses its functional edge.

A Modern Coffee Shop The Blurry Line Between Cells and Silicone

If I were to return to that same coffee shop in 2017, the effects of expanded technical functionality would be quite evident. While in 1921 I stood in a line and struggled to remember the name of a particular pastry, in 2017 I can order ahead using a mobile app on my cell phone and consult the app’s memory for a complete history of my orders. For the purposes of this particular task, my phone stores the needed information almost without flaw, is more consistently available than the typical person, and anticipates my goals and needs in the context of the task. In effect, my phone becomes the ideal partner for transactive memory. Over time, as I become accustomed to ordering through the app, the capacity to order coffee from anywhere becomes integrated into my functionality as a physical-cognitive system, just as years of action and perceptual feedback have resulted in the knowledge that I can fit through the space between the young woman’s chair and the adjoining table.

A similar increase in functionality would enable me to keep in touch with Dan through social media, and to talk with him face to face regardless of where he is. In these cases, my presence reaches beyond the location of my physical body, allowing me to receive perceptual input from around the corner or around the globe. If such functionality were to become transparent and seamlessly integrated into the human cognitive system, the individual’s sense of presence and capacity for action could be expanded indefinitely, further blurring the line between the biological and technological elements of cognition and action.

Flexibility, Security, and Access Will Biological Memory Always Maintain an Edge?

Still, some may argue that knowledge held in biological memory continues to provide unique functionality, principally because, unlike externally stored information, biological memory is not easily lost or destroyed and is not vulnerable to the potential influence of the powerful entities that provide and thus control the technological components of extended cognition. In light of these objections, it might seem that technological information storage poses some of the same problems that plagued ecological memory storage in our 1920’s cafe.

However, it is important to note that these difficulties are not unique to extended cognitive systems. Just as information stored in the cloud would be lost with sufficient damage to the servers on which it is stored, information stored in the brain is just as vulnerable to the effects of trauma. Perhaps more importantly, just as information stored in technological subsystems is vulnerable to manipulation by the entities that control those subsystems, so too is biological memory vulnerable to manipulation at the hands of the individual’s own biases and the contingencies of his or her environment. In a particularly provocative study of rich false memories, Shaw and Porter (2015) used suggestion techniques to generate memories of participants committing crimes. Following interviews, 70% of participants reported committing a crime that never happened, and even voluntarily offered detailed false accounts of their actions. If biological memory alone is this fallible, then perhaps the problems posed by its integration with technological components are less artifacts of technology than they are manifestations of the same cognitive demons that we as a species have been confronting for millennia.

A Functional Symbiosis In Cells or in Silicone, Knowledge is Knowledge

In the long term, when the cognitive channels between man and machine broaden and the imaginary line between the two grows increasingly thin, it’s reasonable to suspect that mankind’s capability for perception, cognition, and action will expand in as yet unimaginable directions. As the functional distinction between human and machine cognition fades away into the past, we will face an era in which the challenges of human life continue to be created and solved in the shared cognitive realm of man and machine. Looking ahead to this future, it seems reasonable to believe that only when we embrace these challenges, fearlessly expanding the previous limits of cognition and action, can we most completely harness the unique power of our cyborg nature.

Obligatory Disclaimer: The opinions expressed in this post are my own, and do not necessarily reflect the positions of any organization with which I have been or currently am affiliated.

Demystifying Density Estimation for Designers