The Documentary Moment in the Digital Age: Establishing New Value Propositions for Public Memory

This paper was originally given as the opening keynote address at the Association of Canadian Archivists' 35th Annual Conference (Halifax, Nova Scotia) on June 10th, 2010.  It was subsequently revised and published as Daniel J. Caron and Richard Brown, "The Documentary Moment in the Digital Age: Establishing New Value Propositions for Public Memory," Archivaria 71 (Spring 2011), pp. 1-20.
 
Introduction
Today, we live in a world that is becoming quintessentially self-documenting. Our lives are constantly recorded through surveillance and sousveillance; we also continuously and instantaneously record  ourselves at work, at play, in public and private conversation, in our business and social relationships
(Twitter, Facebook, MySpace, YouTube, etc.). Our society has become fully conscious and aware of information as a social, economic, and memory utility, and moreover, information resource processing and media literacy have become fundamental to learning, knowledge, and progress. As individuals,
groups, and organizations, we create, provide, ingest, and store information at previously unimagined rates and levels, and, collectively, we are voracious information resource producers and consumers. Now, in the midst of all this production and consumption, two basic questions are emerging in relation to
the current and continuing occupational relevancy of the professions (and their institutions) traditionally associated with the cultural management and preservation of information resources and knowledge, i.e., archivists, librarians, and curators. These are: What is the relevance of these information professionals in such an information-laden and information-intensive society? How will this
relevance be manifested?
 
This essay explores aspects of these questions with reference to an experience familiar to all archivists: the seamless moment of time and space within the remembering process when communities become aware of, and confront, the instability, fragility, and ephemeral nature of memory. This is when society
must make decisions about the communication, capture, treatment, and management of its information resources in relation to perceptions or understandings of their continuing purpose, value, utility, impact, or legacy. This is when “collectivities” must finally, very deliberately and self-consciously,
invest in, and provide for, the preservation of information initially as a social or economic asset, and subsequently, as an accessible civic good of public memory bringing meaning to society over time.
 
To focus our discussion, we will reflect upon the nature and dimensions of this critical decision-making experience, what many archivists would typically term appraisal or acquisition, but what we will call the documentary moment. In particular, we would like to relate the concept of the documentary
moment to the construction of public memory by dedicated institutions within the digital environment.
 
The Evolution of the Documentary Moment
Until very recently, memory institutions like archives have exercised a de facto monopoly over the constitution and mediation of public memory; the operating circumstances of this monopoly have largely allowed archives to approach the identification, appraisal, and preservation of heritage information resources as post-creation or post-production memory functions. Generally, archivists
have been able to wait – sometimes for very long periods of time – until after information has been created or produced and used before applying methodologies, typologies, and criteria to determine its potential status and value as a component of public memory.
 
Through the course of the twentieth century and beyond, a variety of appraisal strategies has emerged, including the application of value taxonomies to “information products” and forms of structural functional and documentary analysis of information creators and their “production contexts.”
 
Among the most sophisticated concepts with successful applications have been documentation strategy, and especially, macroappraisal. Typically, these approaches have been conceived to enable memory institutions to satisfy roles and responsibilities around the provision of continuously evolving memory capacity as a public good; arguably and with few exceptions, most of them
represent post facto forms of cultural or historical mediation. In effect, the substance of their declarations have largely been framed and purposed within cultural models of interpretation, or what might loosely be called the development of “historical memory.” In this sense, they have often been culturally subjective and constructive within the nature of their intention and intervention, and coincidentally, conceptually formulated from the perspective of hindsight, even in instances where memory value is either to be anticipated or predicted through research and analysis.
 
The application of hindsight to the appraisal of information resources in the determination and construction of public memory has been possible for several reasons. First, the information resources under examination have been analog, i.e., information communicated via physical media and objects (e.g., books, textual documents, photographs, audio-visual recordings) that have relative
durability and stability over time. While these media will not necessarily last forever – even paper has its limitations – and conservation technique is required to maintain stability in the longer term, there is an interval of survivability de facto encoded into analog media, which permits a hindsight vantage
point. Second, the volume of the information resources available for memory consideration has been comparatively limited to the extent that there have been opportunities for institutions to let the passage of time factor into the notion of memory value; or there has been time to undertake granular investigations of their potential utility according to taxonomies of subjective and ostensibly
objective criteria; or else there has been time to research and analyze creator contexts and corresponding information production and to ruminate upon, and arrive at, acquisition conclusions. Alternatively, some memory institutions have decided not to make any decisions about the memory value of information resources by simply engaging in comprehensive collecting within
specified domains, or by announcing their institutional intention to do so as an organizational declaration of public memory default. This kind of nondiscriminatory approach to acquisition has been considered entirely feasible within the analog environment.
 
The identification, appraisal, selection, and declaration of analog information resources as civic goods of public memory – in the broadest sense – have essentially been conducted within a long documentary moment of decision making largely because of their relative endurability and limited volume.
These activities have mostly taken place within an anachronistic time and space, in some instances, four or more evolutionary stages away from the first contexts of their human agency or intention (i.e., the identification, appraisal, selection, and declaration of analog information resources is far removed
from what could be called the social sequence of their original significations of situation, experience, and meaning). Within this “long moment” of anachronism, and considering the corresponding literature and ongoing debates, it is fair to say that memory institutions and archivists have struggled with the
concept of memory value, the identification and selection of memory content, and the creation of appropriate memory preservation methodology and criteria. Addressing these questions and issues has been exacerbated by the fact that there is no universally accepted archival appraisal theory to frame these challenges, only local domain theory and practice.
 
This is not so surprising. With the original “event horizons” and social contexts faded or fading over variable periods of time, and corresponding information resources inevitably filtered through various treatments and interventions (i.e., writing, reading, storing, arranging, describing, organizing,
explaining, interpreting, historicizing), a myriad of factors, interpretations, opinions, and views are possible, especially when they are being considered primarily within cultural or historical frameworks. In fact, some institutions have simply resorted to collecting and organizing information sediment about
society as comprehensively as possible within the scale and means of their capacity, or they have developed ad hoc criteria of public memory value on a generational basis linked to existing post facto orthodoxies of contemporary understanding and/or current user interests. Not only has the documentary
moment become very long in memory institutions owing to its largely interpretive and cultural context of operation, but the perception of the intervention as something entirely monumental and fundamentally complex has necessarily positioned institutional decision making around the constitution of public
memory within rigorously detailed examinations of memory residue through selection and other processes.
 
Whether or not these long, contemplative approaches to the documentary moment and its related strategies and methodologies have served public memory well in the past is not at issue here, and opinions would vary on this point in certain instances of application. More immediately important from our
perspective is that the arrival of the digital age has completely transformed the contextual phenomena previously associated with the documentary moment, contracted or reshaped its contemplative time-space, and brought many new factors forward into our decision making around memory value. In the
process, the digital age has effectively undermined many of our assumptions and approaches around the construction and constitution of public memory. In other words, the circumstances and the environment of the documentary moment have substantially changed to the extent that some of our former value propositions and acquisition outcomes are no longer appropriate.
 
This would include the option of simply collecting information resources. For example, initial attempts to apply analog collecting strategies to cyberspace – typically in the form of web-harvesting – are already being called into question for a variety of reasons; memory institutions are now beginning
to recognize the enormously complex scope and scale of the paradigm shift represented by the transition from analog to digital communication. Indeed, the first institutional encounters and attempts to grapple with the Internet as a very large series of publications – because the information involved
was ostensibly in the public domain, and websites were initially considered to be published manifestations of information resource – both fundamentally misread and misunderstood the communications ethos of the Web as a completely new information environment and social dimension, and could not relate and adapt a memory collecting mind-set to its almost constant state
of evolution and metamorphosis. Moving forward, both the dynamic nature of the new “memory marketspace” within networks and digital transformation require a new form of memory archaeology focused on the documentary excavation of contemporary non-physical sites, environments, and networks continuously producing and streaming live information, knowledge, and cultural resources in “timeless cyberspace.”
 
In the digital world, the documentary moment is entirely active, strategic and present in the immediate “now,” yet it potentially also projects and elongates – rather than simply contracts – its time and space forward into continuous decision making and provisioning for preservation, i.e., things may not be as conceptually or contextually straightforward as they might initially appear. In their analysis of cyberspace and its “infosphere,” for example, some observers have noted a transition wherein memory as “registration and timeless preservation” (the Platonic view) is being replaced by “memory as accumulation and refinement,” notably in the Participatory Web 2.0 and Cloud Computing environments. In these environments, time adds value, and information resource applications typically “get-better-by use,” improve with age, or in some instances they diminish in value, the nature and context of value being of participatory dimension rather than linear in evolution. Alternatively, Web 1.0 and the Semantic Web are considered “time unfriendly” in terms of knowledge and memory potential within their corresponding information space and time because of their inherently static and artificial nature. The evolution of the Web in the space-time context of memory is highly complex and enormously challenging. When and where will archivists and institutions intervene in the interests of continuing memory preservation? Are such memory interventions by archivists or others actually required? If yes, what kind of interventions? None of these questions are especially new, but we must now address
them in practical terms.
 
Let us have a very brief look at some of the other elements and factors redefining and reshaping the space and time of the documentary moment in the digital age, and how these will potentially impact and influence the composition and preservation of the civic goods constituting public memory in
the future.
 
Social Transformation
First, a broad merger of technology, economics, information, organizations, and people is leading to social transformation and fundamental changes in the perception and utilities of information and knowledge. The economic and social dimensions of this convergence are clearly of a systemic nature,
with information and communications technology as the common economic denominator and social enabler. In effect we are witnessing the transactions of human activity in all of their variable forms transitioning from a physical to a non-physical dimension of social communication within networks.
 
For memory institutions, among the most significant impacts of this sociotechnology convergence are the increasing demands for information resources and their corresponding commodification and commercialization within a new information resource marketspace; new sets of consumer expectations
around the timely accessibility of information resources; a massive proliferation of information service providers and distributors; and the removal of traditional information resource filters and their substitution with networks of information resource intermediaries, consumers and “prod-users” (or “produmers”). This new information resource environment is both redistributing and complicating the development of public memory far beyond the confines and semantics of analog information resource intelligence and learning experience. It is shifting the context of information resource and memory development from relatively formal, controlled, and ordered relationships to the informal, uncontrolled, disordered, experiences and unlimited communications relativity of cyberspace. And it has effectively ended the public memory monopoly once exercised by archives, libraries, museums, and others.
Part of this new complexity is linked to the nature and dimensions of information itself, insofar as the innovation of digital media and networks is also transforming information. In some limited circumstances, older, traditional forms remain constant and intact, but most of our familiar modes and means of information and communication are undergoing a metamorphosis, and assuming new capacities and utilities. We are also witnessing the genesis and proliferation of wholly new forms of information production and media with no foreseeable innovation end in sight.
 
The primary issue is that archives have little or no experience with “rich” communications and social media, neither in terms of the technology involved, nor in terms of the documentary products being produced, including: their characteristics; the extent of their distribution or the nature of their
repurposing or reuse; and especially not in terms of the potential memory value of the information resource inputs and outputs. To put it simply, there are really no precedents or antecedent reference points for memory value within the digital context of social media. At the same time, archives are
beginning to recognize that: (1) the world of information and communication has almost entirely transitioned into the transactional marketspace of the Internet; and (2) the public memory of contemporary society is also in the process of vacating traditional media to now largely reside in its immediate and corresponding cyberspace. If – given the volume and ubiquitous nature of
digital information – selecting, or collecting, or other analog memory strategies are not viable or feasible within cyberspace, how will institutions adapt and continue to evolve their public memory interventions?
 
Information Creators and Sources
We are also experiencing fundamental change in the relationships between people and information. One of the most significant changes concerns the number of creators actually creating, producing, and distributing information both as a commodity and as a resource, and the impact this is having upon
the evolution of public memory. In information resource development terms, in less than twenty years we have moved from a monopoly of information production and mediation to a virtual oligarchy (first generation Internet information service providers, such as Microsoft, Apple, Yahoo, Google, Amazon,
etc.) to a “here comes everybody” democracy enabled by social media, in which a very large percentage of the population is participating and collaborating in the creation, production, and consumption of digital information. We have also moved simultaneously into machine-generated information sources and resources through self-directed and artificially intuitive tagging and linking
utilities within networks, both not only creating new, but also manufacturing and repurposing, ubiquitous information as information resource derivatives. Transformation is also occurring within the confines of the traditional and familiar relationships between people and information long established
in the intrinsic nature of analog media, and the modality of our interactions with them as basic sources and filters of information, mediation, and meaning. Advances in information and communications technologies are fundamentally altering the way people think about, understand, interpret, assign meaning to, create, use, produce, exchange, receive, store, and provide information. Individuals are also re-adapting – both disconnecting and reconnecting – the way they gain access to each other and to an enormous variety of information, services, and technologies offered by business, government, and communities through networks. Within these processes of social transformation and  sociotechnology convergence, the distinctions of significance, authority, meaning, and value that have been previously assigned to, or established, between information resources based on their status or provenance – for example, whether information is published or unpublished – or the nature of the communications
medium in which it is transmitted or distributed, have all largely become irrelevant to creators, producers, users, and consumers of information in the digital environment. In other words, the value, utility, and mediation of information resources transcend the status, medium, or mode of their creation.
In the digital age, communication is primarily about the production and consumption of “information resource,” and not about the nature or status of the information resource container.
 
For memory institutions, part of the conundrum is that the diversity and multiplicity of contemporary information generators, producers, sources, and containers provide unprecedented access to sources and voices either previously untapped or heretofore unheard of or unacknowledged, and permit the
development of more representative and inclusive public memory across all social sectors. At the same time, the choices are practically unlimited, and the choosing becomes incrementally far more difficult and complicated than it was in the pre-digital era, especially within the epistemological sense of
memory context. Two of the great leavening or democratizing impacts of the digital age will certainly have to be addressed. First, it is no longer a given assumption that the source, status, and medium of information constituted in the form and format of its container (e.g., book, document) automatically
convey authority and value from contemporary social memory perspectives. In fact, users of new media often simultaneously fulfill roles as readers and authors of text to the extent that the distinction between the author and the public “is about to lose its basic character.” Second – and perhaps most
important from our perspective as archivists – the capacity of private individuals to blog, post videos to YouTube, or otherwise broadcast details of their lives, thoughts, or experiences complicates the relationship between individuals and collective memory. How do we approach the notion of collective
memory within interactive media, and how do we accommodate its increasingly participatory nature?
 
Coincidentally, the provenance of information resources has also become highly problematic, insofar as it is the contexts established within particular digital domains and networks that invariably provide information with associative qualification and ambience as public knowledge and memory. Archivists like to say the “context is all.” What are the characteristics of the new ambient creator contexts within networks? Clearly, new value propositions for information resources will need to be developed in relation to the contextual relationships established around information resource development within networks.
 
Information Volume
The development of new relationships between people and information has also led to some largely unanticipated consequences. Among the most important incidental results of digital innovation is a world of superabundant and largely unstructured and undifferentiated information sources and resources.
The increase in the volume of information is both symptomatic and catalytic of a looming information value crisis. Thanks to the Web and its ever- expanding application and content layers, as well as other affiliated networks, communications links, and broadcast-transmission capacities, the world is
now in the midst of what the computing experts are currently calling an “exaflood” (of exabytes), the scope, scale, and dimensions of which are overwhelming regardless of whose calculations one accepts, or the nature of the criteria being used, or in what year the measure was taken. In any instance of
explanation or analogy, the productivity is prodigious and continuously accelerating, and there are very few recognized socio-economic or other determinants generally in place to permit information resource differentiation for the purposes of deciding its continuing persistence, preservation, or disposal
based on criteria of value. Essentially, the Web represents semantic and  epistemological chaos from current public memory perspectives.
 
Information Preservation and Persistence Directly linked to the issue of overwhelming information resource generation and productivity is a fundamental problem of information preservation,
insofar as society’s capacity to create and produce information has far outdistanced both its physical and virtual capacity to store and preserve it; this gap continues to grow exponentially. One of the great myths of contemporary information technology is the notion that society possesses unlimited
information storage. In fact, the production of digital information has already outstripped global server capacity by an estimated factor of four or five. A related myth concerns the costs of information storage. Typically, the issue of storage is viewed in terms of physical capacity, and it is true that
continuing advances in microchip engineering are reducing information storage space to a virtual status approaching the atomic level, and that digital storage containers are becoming far less expensive than they once were. The real cost of information preservation, however, lies not in the physical storage
of data, but in the administration, management, and accessibility of the information objects inside the storage containers – regardless of how big or small – over time, the costs of which are rapidly escalating out of sight. This is especially true in the case of online storage and preservation, which is the expected norm within participatory cyberspace and fundamental to information resource discovery, capitalization, and public utility within collaborative networks, as opposed to near-line storage or off-line “dark storage,” which are progressively less expensive from an accessibility standpoint, though not
necessarily from an administrative and management perspective.
 
Essentially, it is the temporary and medium- to long-term preservation or persistence of digital information objects in accessible form, which represent the real challenge in terms of information resource development and corresponding socio-economic utility. Even if one can assume that most digital objects, however fragile in terms of continuing readability and accessibility, do have a temporary endurability, this does not necessarily elongate the documentary moment and its corresponding decision making around value. On the contrary, given the technological enhancements and metadata markings
necessary from the outset (beginning with the technical engineering phase) to ensure the persistence of information objects in digital cyberspace, decisions about the survivability of information resources will need to be taken before, during, and immediately following the act of creation. Effectively, the value, status, destination, and persistence of digital information resources will need to be determined and decided concurrently (i.e., information resource creation or production processes will need to proceed with greater documentary awareness and consciousness). For public memory institutions, which commonly think in terms of perpetual or everlasting preservation (as opposed to temporary
or perennial preservation), this challenge is of first order importance and magnitude, and potentially turns the world of public memory upside down, both in terms of its theory and practice. It is entirely possible and increasingly logical, for example, to propose that the preservation of public memory
should transpire through various interventions and mediations made within cyberspace, rather than through physical transfers of virtual civic goods from creators to a dedicated repository. Is it possible that public memory could become a network or a persistent “computing cloud” within the Web?
 
The Documentary Moment in the Digital Age
Reflecting upon the nature and substance of the changes currently happening within the information resource environment – which is the elemental wellspring of our capacity to remember – our sense of the growing complexity around the documentary moment is that archives need to fundamentally
reconsider and rethink institutional vision and purpose, and coincidentally, that archivists need to re-examine their professional ethos and objectives. In particular, we need to establish new value propositions around the construction and constitution of public memory, and we need to do this on a collaborative basis with those beyond the traditional memory institution domains.
 
Part of this introspection will involve asking the right questions. For example, given the social and technological phenomena that we have outlined, what are the elements and characteristics of value and significance necessary or sufficient to warrant the preservation of digital information sources and
resources in the form of public memory? How should criteria be established, on what basis, and who should make these determinations and decisions? These are some of the core and essential questions.
 
But we also believe that our lines of questioning must probe more deeply below the surface of our daily business operations. They need to reach down into the very elementals of institutional and professional public memory métier. A crystallizing question was recently posed during a keynote at the
annual conference of the Association of Canadian Archivists: “In the digital environment, does contemporary documentary heritage need to be identified and preserved within a cultural context to be meaningful?” In the past, most archivists would have treated this particular question as a rhetorical statement; after all, the public memory context of the archive has long been rooted in cultural mediation typically framed in forms of historical synthesis or interpretation. This has been self-evident for many years.
 
However, our reading of the current information resource environment – and we are certainly not alone in this reading – is that society has moved well beyond having its public memory mediated, and its culture and history contextualized by dedicated memory specialists. Earlier, we referred to a
phenomenon identified in the literature, wherein information and communications technologies are changing the construction and constitution of public memory because online sites of new media recording and storage represent and enable the translation of memory from the individual to the collective. Like practically everything else, the documentary moment in the digital age is becoming instantaneous, participatory, and collaborative within networks to the extent that the construction and constitution of public memory is also becoming instantaneous, participatory, and collaborative. All of us as individuals potentially see ourselves in public memory, just as most of us are now running private archives at home, automatically and constantly updated in our external hard drives.
 
And so we return to the questions posed at the beginning of our essay. Within this superabundant information resource environment – our contemporary and future source of public memory – what is the relevance of archives and archivists, and how will this relevance be manifested? What spaces will
the public archive and public memory occupy in an information ecosphere dominated by new information service providers, consortiums, and consumers? This is an environment where corporate giants like Google and Microsoft, using the unimaginable mass of their stored data and huge distributed computing
power, are already conducting temporal trends analyses to predict and analyze future events. Imagine: Google’s application will be called Recorded Future. In effect, Google is already predicting and elongating forward the next generation of documentary moments. It is anticipating the future, and identifying the documents that will be necessary to understand it.
 
In all of this, our sense of the documentary moment in the digital age is that archives and archivists, in conjunction with others, those both within and beyond the information resource domains, can continue to play prominent roles and fulfill key responsibilities both now and into the future. But we also
think that these roles and responsibilities will need to be focused, formulated, and implemented differently, with substantially different goals, objectives, and results in mind. Within this context, we not only need to be asking the so-called “right questions,” but also different ones, certainly different from the ones we have traditionally asked in the past. For example, do we need to establish a new documentary framework and new value propositions for public memory to be preserved within memory institutions? Does this documentary framework continue to be informed broadly by cultural and historical interpretive syntheses, or does it move us toward a different and potentially more concentrated sphere of intervention?
 
One option could be a more direct focus on the public endowment and preservation of our foundational civic goods – the original documents of our decisions and actions, and the information in our books and other documentary media and artifacts – that are required within society to articulate, express and share common goals, assumptions, values, and ethics; to provide individuals and groups with the capacities of social literacy necessary to enable their democratic participation within communities; and to ensure accountable public administration and responsible governance under the rule of law: in effect, we mean focusing attention on the preservation of society’s causa materialis, the documents that permit us to socially live our lives within a state of law, to function collectively as a democracy, and to have continuing and inclusive social consensus and progress through the distribution and sharing of information resources, and the preservation of an accessible public memory. In other words, we are suggesting that public memory institutions should concern themselves primarily with the identification and survivability of the information resources and documents articulating the modern, democratic
state and its broader domain of inter-sectoral governance and activities, including its corresponding regularities, ethics, and discourses expressed through contemporary socio-economic actions and behaviours at various individual, group, and organizational levels. This would necessarily move archives
and archivists away from their traditional preoccupations with cultural mediation, interpretation, and the integration of selected information resources into historical collections of memory, and toward the construction and constitution of a more “civic” public memory serving multiple socio-economic utilities.
 
More practically in the sense of application, would new value propositions supporting such a documentary framework separate us from previous information value taxonomies and traditional methodologies? Would all of this only apply to information resources that are “born digital?” Given the current economics of the public memory business, what kind of other decisions do we need to make? For example, how do we intend to make public memory sustainable over time? What is the nature and constitution of sufficient public memory? How can it become more representative and inclusive of diversity? Many questions asked, and unfortunately few concrete answers thus far, although there are some encouraging signs. Library and Archives Canada (LAC) is currently engaged in a process of institutional modernization, a significant component of which is a reconsideration of the meaning of
documentary heritage in its enabling legislation. This exercise may result in substantive changes in emphasis and direction, notably in relation to the prevailing philosophies, mechanisms, and expected results of appraisal and acquisition activities, which ultimately establish and represent public memory
at the federal level of its constitution. LAC is also engaged in a series of discussions with colleagues from a number of institutions and jurisdictions across the country to establish a way forward through the labyrinth of digital information resources. Together, they have been talking about some of the issues
 raised in this essay, variously at the levels of occupational ideology, philosophy, strategy, application and implementation, and there is a growing understanding and willingness to come to some basic propositions around how the challenges of the digital environment can begin to be addressed.
Of several things we are certain. There is a growing consensus that the public memory challenges of the digital age need to be met by collaborative strategy and research; that potential solutions and interventions will not succeed through independent unilateral actions, but can emerge through
institutional and occupational convergences. And we are all beginning to realize that this collaboration cannot be confined to ourselves as memory professionals and memory institutions. We are beginning to understand that the construction and constitution of the civic goods of public memory are
a collective, social responsibility requiring broad participation across all sectors.
 
As individuals, groups, and organizations, we all inhabit and share the documentary moment in the digital age; it is time to work collaboratively to explore its emerging nature, dimensions, and implications formulated as new public memory.