Digital Libraries and Virtual Universities
Edward A. Fox
Department of Computer Science
Blacksburg, VA 24061-0106, USA
Invited presentation for
"Information research for
designing and planning virtual universities"
Centro Universitario de Investigaciones Bibliotecolgicas
Universidad Nacional Autnoma de Mxico
Cd. Universitaria, Mxico, D.F.
(Library and Information Research Center,
National University of Mexico)
August 11-15, 1997
Universities need libraries. Virtual universities need digital
libraries. From these observations we see that
if we are to
successfully develop virtual universities,
it is important to
understand about digital libraries, and to learn from experiences in
applying digital libraries to improve education
1.1. Digital Libraries
In 1965, JCR Licklider explored the future of libraries, that would
help deal with the information explosion through the use of computer
and communication technologies [LICK65]. By 1997, digital libraries were
becoming a practical reality [LESK97]. As electronic publishing
technologies evolve, authors can create works that will go directly
into digital libraries, rapidly adding quality works to the ever
growing collection of materials available through the WWW.
In the period 1991-1993, interest grew in the U.S. regarding
electronic or digital libraries [FOXE93c]. Early work set the stage for
later studies, by considering interface issues, economic models,
application of protocols like Z39.50 for interoperability and federated
searching, and access to published journals [FOXE93a].
Investigators in the public and private sectors began to identify
requirements, propose architectures, and develop systems [GLAD94].
By 1994 it became clear that a great deal of research was required for
these to be intelligent and useful [FOXE94a], and the U.S. led other
nations into extensive funding through the
NSF/DARPA/NASA Digital Library Initiative
Scores of projects and studies proceeded worldwide [FOXE95a].
1.2. Overview of Two Projects
At Virginia Tech, there has been long standing interest in information
retrieval, library automation, multimedia, electronic publishing, and
since 1991, digital libraries. Some of this work is based in the
NSF-supported Information Access Laboratory [FOXE96a].
Starting in 1987, Virginia Tech became involved in exploration of
electronic theses and dissertations, working with SoftQuad in 1988
to develop a Document Type
Definition to facilitate SGML encoding.
Pilot efforts continued [DALA93], eventually leading to a large
worldwide initiative discussed in Section 2 below. If large numbers of
theses and dissertations become accessible electronically, they should
play a strong supporting role for distance education, especially at the
graduate level, and help extend the possibilities of virtual
In 1991, Virginia Tech received support from NSF to pilot test
digital library concepts in the computer science area, with assistance
from ACM. By 1997, leading computing associations began offering
their publications through digital libraries: ACM at
and IEEE CS at http://computer.org/epub/,
and many others with similar plans.
In the realm of technical reports, computer science efforts
[FOXE95b] harmonized in 1995, leading to the widely used
Networked Computer Science Technical Report Library at
In 1993, additional NSF support was provided to Virginia Tech to apply
digital library technology to improve computer science education;
this is discussed in Section 3 below [FOXE95c, FOXE96b].
2. Networked Digital Library of Theses and Dissertations
In 1996, development began on
the Networked Digital Library of Theses and
(NDLTD - http://www.ndltd.org),
funded by Southeastern Universities
Association, US Dept. of Education, IBM, Adobe, Microsoft, and others.
In addition to the extensive online documentation, there is an article
in D-Lib Magazine [FOXE96c] at
as well as a videotape, CD-ROM, and other supplemental information.
Key efforts have focused on training students, working out policies
and approval mechanisms so that student and faculty interests are
represented, working with publishers and publisher associations to
harmonize the evolution of complementary genre for scholarly
publication, automating the submission and workflow processes for
handling theses and disserations, and implementing solutions for
federated digital library access.
From these efforts we have learned:
- universities can and are forming the foundation of a distributed
service that will aid (graduate and other) education;
- universities have significant infrastructure to make this
possible, though there will continue to be need to maintain and expand
- making this work even more effectively requires solving problems
that might be classified as relating to building virtual
Since there are online talks about the project at
http://www.ndltd.org/talks/index.htm, and since it is clear that
having hundreds of thousands of electronic theses and dissertations will
be of great value for virtual universities, this paper shifts to the
second project area, about computer science.
3. Interactive Learning
with a Digital Library in Computer Science
In 1993 principal investigators N. Dwight
Barnette, Edward A. Fox (director), H. Rex Hartson, JAN Lee, and
Clifford Shaffer began to transform computer science education at
Virginia Tech by applying digital library and other related
technologies, building on a rich campus infrastructure as well as the
Blacksburg Electronic Village
community networking effort.
3.1. Overview of Goals and Objectives
Key concepts of our project
[FOXE95c] are to improve CS education by increasing interactivity and
use of a digital library. The main objectives/accomplishments were to:
expand the content and software (especially interfaces [FOXE93b,
WAKE95]) initially developed with NSF support of our "Envision" digital
library project, "A User-Centered Database from the Computer Science
Literature" [HEAT95]; develop/apply algorithm visualization tools that
are easy for instructors to use in supplementing courses, and feasible
for students to work with as an aid to program development and debugging
[YANG95, SHAF96a, SHAF96b]; incorporate use of specialized digital
library systems like Netlib into related courses; add new courses
related to human-computer interaction, multimedia, and a freshman level
introduction to Networked Information; significantly change courses like
"Computer Professionalism," to make use of interactivity (e.g.,
asynchronous online debates) and digital library support (e.g., adding
to a large History collection); and apply the key concepts to improve
3.2. Current Status and Accomplishments
In 1991 Virginia
Tech began working with ACM through support from NSF on a "User-Centered
Database from the Computer Science Literature" [HEAT95]. In 1993,
Virginia Tech expanded its work on digital libraries to launch the NSF
EI project, partnering with Norfolk State University, which has
developed extensive sets of laboratory manuals. Over 45 courses are
available through WWW, leading to over 5M accesses since 1995. There
are several gigabytes of ACM publications available.
Several courses have all the on-line materials required for self-study
available, and new programs are under development for distance learning
and continuing education. In the new multimedia course, there was a
dramatic increase in bandwidth required for the 1996 offerings as
compared to the 1995 ones, because of more images, digital audio, and
digital video. Due to the development by Prof. Lee (editor of Annals of
the History of Computing) of one of the largest repositories on computer
history, with a unique image collection of the founders and early
systems in our field, there is extensive additional traffic from
throughout the nation. In 1996, with the help of NSF-funded digital
video capture and editing facilties, audio annotations, digital video
movies, and animations to show interactive applications were added.
One of the courses developed under this effort, and extended through
support from SUCCEED, is CS1604, an introduction to networked
information. A self-study version of this course was finalized later
in 1996, and is expected to be widely used throughout the Southeast and
beyond by those interested in a freshman or beginner-level orientation
to Internet, digital libraries, collaboration technologies, etc. This
version has numerous audio and movie files to help learners, an
automated real-time feedback facility (using our SGML-based QUIZIT tool
[TINO96]), and a variety of illustrations and demonstrations. In one old
and two new courses, we have adapted Keller's Personalized System of
Instruction to our networked environment. Students proceed at their own
pace, study on their own, get help through asynchronous communication
with peers and instructors, and in general have much greater flexibility
in learning. Many students prefer this type of course, and in the case
of CS1604 we simply could not accomodate the demand any other way, in
this time of scarce resources. However, students have requested that we
add interim deadlines, since they tend to procrastinate and require help
with time management - this seems to solve the major problem faced
3.3. Materials that Have Been Developed
result of our effort is the prototype Envision system. Its interface, if
ported to Java, and connected to Z39.50, could be a very convenient
means for accessing a variety of bibliographic collections, as well as
richer digital libraries. A second result is the content converted from
ACM. The most convenient portion is several hundred articles from CACM
available now for those with permission using the Dienst system. A third
result is the software created in increase interactivity of learning:
SWAN and QUIZIT. Finally, there are about 10,000 WWW pages of CS
Ongoing collaboration with Norfolk State University (NSU) has led to an
increase in the use of laboratories to aid learning of CS students at
Virginia Tech, and adaptation of many of the Virginia Tech materials and
tools for use at NSU. Another systematic extension has been facilitated
by additional funding from NSF through Southeastern University and
College Coalition for Engineering EDucation (SUCCEED). The SUCCEED
Coalition Grant "Using Computers and Networked Information: Distance
Learning with Networked Multimedia" is expanding through the use of
digital video/audio tutorials, an alternative VRML (Virtual Reality
Markup Language) interface, multiple graphic pathways, an interactive
collaboration medium for synchronous communication, and an online
interactive real time testing component. Outreach work with a number of
universities in the region is underway to help them apply this course to
help them deal with increased interest in this field, and to more
closely approach full "Information Literacy."
3.5. Evaluation Activities
Our evaluation involves typical traditional methods, e.g., pre- and
post-tests, surveys, and focus groups. We performed usability studies of
tools we developed or applied, and used formative evaluation methods to
refine both our tools and courseware. Yet, our project still requires
additional approaches to evaluation. The investigators in our project
are instructors who changed their allocation of time, behavior,
pedagogy, course materials, and tools. To understand the effects of
these changes, ethnographic practices are of great value -especially
regarding use of asynchronous communication (i.e., online debates)
[LAUG96]. Another shift in our evaluation has been to rely on network
monitoring, logging, and analysis. Here we draw upon special tools for
this purpose [ABRA95]. Part of this work has helped improve our quality
of service through caching. The rest has helped us understand
what students really do, what course materials are accessed, how use of
multimedia effects network traffic, and how both remote and local
accesses increase over time. There has been a gradual increase in both
remote and total access counts, if we ignore the valleys occuring during
mid-semester, summer, and end-of-year breaks.
3.6. Benefits Seen and Expected
In summary, we have developed tools, expanded our digital
library systems and content, and built almost 5000 "pages" of
WWW-accessible courseware, increasing the interactivity and quality of
learning about computer science. Evaluation has shown that learning
practices have changed, most students are happy with the emerging
infrastructure and pedagogy, and there is steady growth in access to our
server. Remote users now account for about one-third of the page
requests, adding to the hundreds of students served locally. Our work
has helped train hundreds of students, has aided the work of instructors
interested in teaching courses for which we have developed useful
materials, has developed tools (e.g., SWAN, QUIZIT) that can increase
the interactivity of learning about computer science, and has helped
with the construction of digital library systems and a content
collection in CS (with ACM publications as well as technical reports).
The two digital library projects discussed serve as exemplars
regarding how digital libraries can improve education. Both involve
many sources of information, and because technical reports, theses and
dissertations are prepared and archived in a distributed fashion,
demonstrate how a virtual university service can evolve.
We hope that these ongoing efforts themselves will be widely used, and
that insights from these efforts will be helpful as similar projects
- ABRA95 M. Abrams, S. Williams, G. Abdulla, S. Patel, R.
Ribler and E. Fox. Multimedia Traffic Analysis Using Chitra95. In Proc.
3rd Int. Multimedia Conf. and Exhibition, Multimedia '95, San Francisco,
Nov. 5-9, 1995, 267-276.
- DALA93 K. Dalal and E. Fox. Document Translation:
Dissertations and Technical Reports, TR-93-31, VPI&SU Computer Science
Dept., Sep. 21, 1993, Blacksburg, VA.
- FOXE93a E. Fox and L. Lunin. Introduction and Overview to
Perspectives on Digital Libraries. Journal of the American Society for
Information Science (JASIS), Sept. 1993, 44(8): 441-443.
- FOXE93b E. Fox, D. Hix, L. Nowell, D. Brueni, W. Wake, L.
Heath, and D. Rao. Users, User Interfaces, and Objects: Envision, a
Digital Library. JASIS, Sept. 1993, 44(8): 480-491.
- FOXE93c E. Fox, ed. Sourcebook on Digital Libraries: Report
for the National Science Foundation, TR-93-35, VPI&SU Computer Science
Dept., Dec. 1993, Blacksburg, VA. Available by FTP from directory
pub/DigitalLibrary on fox.cs.vt.edu
- FOXE94 E. Fox. How to make intelligent digital libraries.
In Methodologies for Intelligent Systems, Proceedings of the 8th
International Symposium, ISMIS'94, Charlotte, NC, Oct. 1994. Lecture
Notes in Artificial Intelligence 869, Springer-Verlag, Berlin, 27-38.
- FOXE95a E. Fox, R. Akscyn, R. Furuta, and J. Leggett. Guest
Editors' Introduction to Digital Libraries. Commun. ACM, Apr. 1995,
- FOXE95b E. Fox. World-Wide Web and Computer Science Reports.
Commun. of the ACM, Apr. 1995, 38(4):43-44.
- FOXE95c E. Fox, N. Barnette, C. Shaffer, L. Heath, W. Wake,
L. Nowell, J. Lee, D. Hix, and H. Hartson. Progress in Interactive
Learning with a Digital Library in Computer Science. Invited paper for
Proc. ED-MEDIA 95, World Conference on Educational Multimedia and
Hypermedia, Graz, Austria, June 17-21, 1995, pp. 7-12.
- FOXE95d E. Fox and L. Kieffer. Multimedia Curricula, Courses
and Knowledge Modules, ACM Computing Surveys, Dec. 1995, 27(4): 549-551.
- FOXE96a E. Fox. Virginia Tech Department of Computer Science
Information Access Laboratory. ACM SIGIR Forum, Lab Report Special
Section, 30(1), Spring 1996.
- FOXE96b E. Fox. Digital Libraries, WWW, and Educational
Technology: Lessons Learned. Invited paper for Proc. ED-MEDIA 96,
World Conference on Educational Multimedia and Hypermedia, Boston, MA,
June 17-22, 1996, 246-251. See also
- FOXE96c E. Fox, J. Eaton, G. McMillan, N. Kipp, L. Weiss, E.
Arce, S. Guyer. National Digital Library of Theses and Dissertations: A
Scalable and Sustainable Approach to Unlock University Resources. D-Lib
Magazine, ISSN 1082-9873, Sep. 1996.
- GLAD94 H. Gladney, E. Fox, Z. Ahmed, R. Ashany, N. Belkin,
and M. Zemankova. Digital Library: Gross Structure and Requirements:
Report from a March 1994 Workshop. Digital Libraries '94, June 19-21,
1994, College Station, TX, ed. J. Schnase, J. Leggett, R. Furuta, T.
- HEAT95 L. Heath, D. Hix, L. Nowell, W. Wake, G. Averboch, and
E. Fox. Envision: A User-Centered Database from the Computer Science
Literature. Commun. of the ACM, Apr. 1995, 38(4):52-53.
- LAUG96 S. Laughton. An ethnographic study of Internet-based
applications in education. Ph.D. Dissertation, Dept. of Computer
Science, Virginia Tech, Blacksburg, VA, May 1996.
- LESK97 M. Lesk. Practical Digital Libraries: Books, Bytes,
& Bucks. Morgan Kaufmann, San Francisco, 1997.
- LICK65 JCR Licklider. Libraries of the Future. MIT Press,
Cambridge, MA, 1965.
- NOWE94 L. Nowell, E. Fox, L. Heath, D. Hix, W. Wake and E.
Labow. Seeing Things Your Way: Information Visualization for a
User-Centered Database of Computer Science Literature, TR-94-06, VPI&SU
Computer Science Dept., Jan. 1994, Blacksburg, VA.
- NOWE96 L. Nowell, D. Hix, R. France, L. Heath, and E. Fox.
Visualizing Search Results: Some Alternatives to Query-Document
Similarity, in Proc. SIGIR'96, Zurich, Switzerland, Aug. 18-22, 1996.
- SHAF96a Shaffer, C.A., Heath, L.S., and Yang, J., Using the
Swan Data Structure Visualization System for Computer Science Education,
Proc. SIGSCE '96, Philadelphia PA, February 1996.
- SHAF96b C. Shaffer, L. Heath, J. Nielsen, and J. Yang. SWAN:
A Student-Controllable Data Structure Visualization System. In Proc.
ED-MEDIA 96, World Conference on Educational Multimedia and Hypermedia,
Boston, June 17-22, 1996, 632-637.
Tinoco L.C., Fox E. A., Ehrich R. W., Fuks H. 1996. QUIZIT: An
online quiz system for WWW-based instruction. In VII
Proceedings of the Symposium of Educational Technology: Belo
Horizonte, MG, Brazil, Nov 1996.
- WAKE95 W. Wake and E. Fox. SortTables: A Browser for a
Digital Library. In Proc. 4th Int. Conf. Information and Knowledge
Management, CIKM '95, Baltimore, MD, Nov. 28 - Dec. 2, 1995, 175-181.
- YANG95 Yang, J., Shaffer, C.A., and Heath, L.S., SWAN -- A
Data Structure Visualization System, Proceedings of Graph Drawing '95,
Passau Germany, pp. 520-523, September 1995.