![[CoOL]](/icons/sm_cool.gif){kind=icon}
Since the early 1980s, staff at the National Archives and Records Administration (NARA) have monitored developments in optical media storage technology in order to understand how best to ensure the long-term usability of records of federal agencies stored digitally on optical media.
Three types of optical media can be used to store digital information--CD-ROM, WORM (Write Once, Read Many), and Rewritable. Because these three types of optical media use essentially the same technology to read digital information, the fundamental difference among them is how the information is written or stored. Typically, the production of CD-ROM (4.72 inches in diameter) involves a mastering process that produces multiple copies. In WORM technology, a laser beam and very powerful optics are used to record chunks of information on a single disk. A recent development in CD digital technology called CD-R, that uses the same approach as WORM technology, makes it possible to record on 4.72 inch disks that conform to international standards for physical and logical file characteristics. Information on CD-ROM, CD-R, and WORM media cannot be erased or revised, in contrast with rewritable optical media on which digital information can be changed and deleted almost without limit.
CD-ROM, CD-R, and WORM technologies inherently are attractive for long-term storage of digital information because they can not be erased or revised. Therefore, NARA's interest in optical media technologies focuses largely upon non-rewritable optical media, because of concerns about the long-term usability of digital records stored on CD-ROM, CD-R, and WORM optical media. An examination of the long-term usability of digital records stored on CD-ROM, CD-R, and WORM optical media involves consideration of three critical problem areas: (1) the life expectancy or longevity of the optical media, (2) the capacity of the computer system to measure and compensate for data degradation, and (3) a technology migration strategy that crosses information technology generations. Consequently, NARA initiated a three-part research project through its Technology Research Staff to address these three critical problem areas.
Typically, WORM optical media manufacturers claim five years of shelf life for blank disks and twenty to thirty years of life after recording. These life expectancy claims are based upon test procedures that vary from one manufacturer to another. The absence of generally accepted test procedures for evaluating the life expectancy of WORM optical media means that comparing vendor claims for longevity is like comparing apples and oranges. In 1987 NARA contracted with the National Institute of Standards and Technology (NIST), then the National Bureau of Standards, to develop a standard test methodology for assessing the life expectancy of WORM optical media. The results of this of project are reported in the NIST study, Development of a Testing Methodology to Predict Optical Disk Life Expectancy Values, issued in February 1992. The study proposes a generalizable test methodology that can form the basis for a national and international standard.
Although a standard test methodology for predicting the life expectancy of WORM optical media is very important, it does not address the equally important needs for users to have guidance on the care and handling of optical media. Consequently, in 1990, NARA commissioned an on-going project with NIST to produce a report on the care and handling of optical media. One goal of this study is to identify and develop standardized measurements to verify periodically any degradation in the quality of the recording. As part of this project and with support from other federal agencies, NIST has organized a working group composed of users and vendors that is focusing upon this problem.
Digital data errors can be introduced by the communications system transporting data from one place to another, by the mechanical systems writing and reading the data onto media, by deformations in the media such as spots or micro-level warping, and a host of other causes related to the storage media. From a narrow storage perspective, a primary factor influencing the number of data errors is the storage density of the medium. For example, current magnetic media generally have a storage density of about 50 to 60 million bits per square inch, while optical media store on the order of 150 to 400 million bits of data per square inch by utilizing a laser beam focused to approximately one micron to record and read digitized data. The close tolerances for spacing bits, tracks, and sectors on optical media place heavy constraints upon the positioning mechanisms of optical disk drives. A tracking error of one-half micron (approximately 1/50,000th of an inch) in an optical disk system is enough to cause a stored bit to be read incorrectly. In contrast, magnetic media and systems have much larger tolerances and the possibility of errors occurring when reading data is much lower.
Regardless of the medium, storage of digital information has always included some kind of error detection and error correction mechanism so that data can be retrieved error-free. A number of utility programs have been written for magnetic disk based systems to help users determine the location of these errors, to relocate data to other areas of the disk, and to reconstruct the data that has become partially damaged. Unfortunately, similar utilities suitable for the general user do not exist for optical media.
The close mechanical tolerances in optical media and systems require very powerful error detection and error correction schemes to ensure reliability of retrieved data. Optical systems typically provide a statistical probability of error of only one byte out of every one billion bytes. The application of error detection and error correction schemes to achieve this level of reliability is automatic and transparent to users. However, the error correction schemes are limited to handling error rates below five out of every 10,000 bytes. Once this limit has been exceeded, the error correction scheme can no longer compensate for or guarantee correction of all errors, and the optical medium essentially becomes useless.
One solution is to have drive electronics that are capable of providing access to error detection/correction data so that monitoring techniques can be used to monitor the gradual degradation of the media before the level of errors becomes catastrophic. Utility programs could be written to capture this information on a periodic basis and provide the user with a profile of the optical media. The NIST working group mentioned earlier expects to produce a report in November 1992 that will identify this and other possible solutions. The report is expected to encourage industry cooperation in the development/modification of optical drive error reporting systems so that optical drives from different manufacturers will all have the capability of supporting a common set of error monitoring and reporting utilities. Eventually the industry agreement would become an international standard.
The third crucial problem affecting the long-term usability of digital records created by federal agencies is the failure to develop a migration strategy for moving records to new media and technologies as older ones are displaced. The unavoidable fact is that digital records are technology dependent and therefore technology obsolescent is likely to be the most serious impediment to the long-term usability of digital records. Therefore, the development and implementation of a migration strategy to ensure that digital records created today can be both processed by computers and intelligible to humans in the 21st century is absolutely essential.
In the NARA study, _Digital Imaging and Optical Media Storage Systems: Guidelines for State and Local Government_, completed in late 1991, the broad characteristics of a viable migration strategy were outlined. A follow-on study of digital imaging and optical media storage systems with guidelines for federal agencies, scheduled for completion near the end of 1992, will explore in greater detail alternative migration strategies.
The research and investigation into these three critical problem areas for optical media is part of NARA's on-going information technology research activities to address problems of electronic records, a matter of great concern to the entire archival community. Consequently, the findings and conclusions reached through NARA sponsored information technology research are intended to be shared with the entire archival and professional community