May-June 2004

Internet References Unreliable


Internet pages cited as references in scholarly journals tend to disappear over time, leaving supplementary information inaccessible, according to a study by researchers at the University of Colorado that was published last fall in Science magazine.

The study, titled "Going, Going, Gone: Lost Internet References," reviewed the references cited in articles appearing in three major publications: The New England Journal of Medicine, The Journal of the American Medical Association, and Science. Internet references accounted for only 2.6 percent of all references in a sample of more than a thousand articles published between 2000 and 2003, but 30 percent of the articles had at least one Internet reference. The researchers found that after two years, up to 13 percent of those references were "bad links," meaning that users trying to gain access to them received error messages instead.

Bad links were most frequent among addresses ending in ".com," and least frequent in addresses ending in ".org." Internet addresses may become bad links when an organization shuts down its Web site, takes materials offline, or changes the addresses it uses (often, ironically, in an attempt to make information easier to find). In addition, since "no consensus on Internet reference format exists," it is difficult to ascertain how long ago the author of a publication may have viewed the Internet site referenced.

The study notes that as use of Internet references rises, the percentage of bad links will likely climb as well. The researchers conclude that an urgent need exists for new policies for documenting and archiving Internet information used for scientific research, such as requiring scientists to submit a printed hard copy of referenced Internet information.