JISC Digitisation Programme

There are still a few places left for the event Digital Impacts: How to Measure and Understand the Usage and Impact of Digital Content, 20 May, Oxford

Digital Impacts will discuss methodologies for measuring the impact of digitised resources and embedding them in teaching and research; it will present projects from the JISC-funded Impact and embedding programme, which provide cases studies on the impact of resources as wide ranging as institutional podcasts, multimedia dance resources, historical sources and learning resources repositories and will lanuch the newly updated Toolkit for the Impact of Digitised Scholarly Resources.

Speakers include Brian Kelly, UKOLN, Prof David Robey, Oxford eResearch Centre (OeRC), Melissa Heighton, Oxford University Computing Services, Dr Jane Winters, Head of Publications, Institute of Historical Research, representatives from the projects, and the workshop convenors, Dr Kathryn Eccles and Dr Eric T Meyer, both from the Oxford Internet Institute.

More information about the event, a draft programme and registration form are available on the Digital Impacts web site.

JISC has recently funded seven new projects to explore the Impact and Embedding of Digitised Resources.

The aims of the programme are to:

• Facilitate institutions in carrying out an analysis of the impact of their digitised resources/collections that have been live for at least one calendar year.

• To develop strategies and practical solutions to ensure the increased use and impact of the resources in teaching, learning and research within higher education (HE)

The Projects

Below is some further information about the new projects.  This will be updated shortly with a JISC webpage for the projects and links to project webpages.

British History Online as a Case Study

Institute for Historical Research, University of London

This proposed project will use the Toolkit for the Impact of Digitised Scholarly Resources to enhance and broaden the British history Online (BHO) information on usage and impact of digital resources, thereby informing future development of BHO and contributing to its long-term sustainability and use in research, teaching and learning.

Dance Teaching Resource & Collaborative engagement Spaces (D-TRACES)

Coventry university

The D-TRACES Project (Dance teaching resource and collaborative engagement spaces) will exploit a unique and significant digital dance resource, the Siobhan Davies digital archive. Following a systematic analysis of user engagement and impact on the local student experience, the project will develop a model for embedding the digital archive within the Personal Development Planning (PDP) element of the undergraduate dance curriculum at Coventry University, thereby generating learning objects for much wider distribution.

Listening for Impact

University of Oxford

This project will perform a thorough, rapid analysis of the impact of the public Oxford Podcast audio video collection of 1800 scholarly items, launched in September 2008. By mixing technical innovations and user engagement it will increase discoverability and reuse of material within teaching, learning and research.

Embedding a vision of Britain through Time as a resource for academic, research & Learning

University of Portsmouth

A Vision of Britain through Time may be the worlds best local history web site, but in no way meets academic expectations for an on-line GIS: a comparison with the web sites created by the US National Historical GIS shows almost no overlap in functionality. This project will add enhanced statistical mapping, a custom mapping facility, and new data download facilities covering historic mapping, boundary maps and, crucially, statistics. These facilities will complement not duplicate existing download facilities at Edina and UKDA. Access to most new facilities will be Shibboleth controlled and restricted to UK HE users, to manage computational load and for copyright reasons.

The project will create a detailed report on a site with high and unusual usage patterns, and unusual success at income generation. One goal is simply to better measure specifically academic use.

SPHERE

Kings College, London

Stormont Parliamentary Hansards Embedded in Research and Education (SPHERE) will attempt to extend the work of LAIRAH and similar projects by developing new methodologies for assessing the value of digital resources, and will implement a series of measures to assess the use, value and impact of the digital scholarly resource, the “Historical Hansards”, and implement a series of practical approaches to embed the resource within teaching, learning and research.

Crime in the Community: Enhancing User engagement for Teaching & Research with the Old Bailey Online

University of Sheffield

The Old Bailey Proceedings Online is accessed by a wide community, but academic users have to date not fully exploited this resource and its advanced functionality in their teaching, learning and research. Crime in the Community will assess the ways in which this website is currently used, and generate a series of new tools and online facilities that will allow educationalists and researchers to make more effective use of the 120,000,000 words of highly tagged and accurately transcribed historical text available through the site.

HumBox Impact

University of Southampton

The JISC/HEA funded HumBox project developed a repository of OER materials for the humanities. The project was a collaboration between four Humanities HEA Subject Centres (LLAS, English, History and Philosophical and Religious Studies), and worked closely with the wider UK humanities community to establish what is now a flagship example of what can be achieved in a discipline through OER engagement.

The HumBox Impact project will undertake an analysis of the impact of the collection on contributors and the wider teaching audience and will investigate emerging working and sharing patterns.  HumBox Impact will use the findings from its study to: Develop strategies to ensure the increased use of the HumBox collection in HE; and develop tools (web site enhancements) to support new and emerging working patterns.

Having recently issued our Funding Call on impact & embedding of digitised resources now seems an appropriate point at which to reflect on some of the work JISC has done to investigate and facilitate the impact and usage of digital resources.

The Oxford Internet Institute (OII) recently submitted their Final Report on a workshop they undertook entitled: Digital History Workshop: Connecting Researchers to Digital Collections .

The workshop addressed some of the issues and outcomes from the OII’s JISC funded study: The Usage and Impact of Phase I Digitisation projects and Toolkit for the Impact of Digitised Scholarly Resources (TIDSR).

These issues can be broadly defined as:

• The difficulty in connecting with potential users (researchers, teachers, students and the public);
• In some disciplines (i.e. History), habits dictate that researchers and students are not enthusiastic about the use of online resources for their studies.

The workshops, undertaken over two days, attempted to focus on scholars and researchers who had an interest in learning how digital resources could enhance and potentially transform their research and work.

Focussing on early stage researchers in the second event, the workshop attempted to establish, and challenge attitudes to digital resources early on in the researchers career.

The workshops attempted to confront some of the barriers that are often noted in the uptake and continued use of these resources:

“Of key importance to these workshops, but in particular to the first workshop, was the combination of information sessions with reflective papers from key scholars working in various research areas, and the opportunity to ask questions both of custodians/creators of digital resources and of those already using these tools for academic research and teaching.”

Conclusions

By having prominant scholars demonstrating their use and research activities in connection with online scholarly resources, these workshops highlighted that:  “Researchers are attracted to new methods and approaches most readily when presented with tangible and substantive examples from their peers and mentors“.

Furthermore, the TIDSR study and workshops highlight the importance of having multiple methods for seeking out information about users.  This user research can help deepen our understanding of how these resources are used and embedded in the practices of teachers, students and researchers.

Final Thoughts

“We found during the TIDSR project that funding for monitoring usage and impact beyond the launch of digital resources is often limited, and it is therefore vital that such funds are put to the best use.”

Hopefully the funding for the 7/10 grant call: Impact and Embedding, can help address this recommendation/concern from the workshops and wider impact study.

JISC has just announced funding for its Grant call 7/10: e-Content and Digitisation programme: Impact and Embedding of digitised resources.

Funding of up to £150,000 is available for projects addressing the impact and embedding of digitised resources.  It is anticipated that 4-6 projects will be funded and the maximum funding for any one project is £40,000.

Proposals are not limited to previously funded JISC projects.

The deadline for receipt of proposals in response to this call is 12 noon on Friday 9 July 2010.

The call aims to:

• Facilitate institutions in carrying out an analysis of the impact of their digitised resources/collections that have been live for at least one calendar year.

• To develop strategies and practical solutions to ensure the increased use and impact of the resources in teaching, learning and research within higher education (HE)

This call is in response to a number of important studies and pieces of work attempting to asses the impact and usage of digital resources, including the impact study carried out on the phase I JISC digitisation projects.

Projects must start in October 2010 and complete by March 2011.

The Toolkit for the Impact of Digitised Scholarly Resources (TIDSR), developed for JISC by the Oxford Internet Institute, is now available online for everybody to use.

If you have been struggling with making sense of hits, visitors numbers, log analysis, users feedback, wondering how to interpret all this data, how to gather it in the first place and how to assess whether your resource is being used and how, the toolkit will provide a clear, concise and easy to use framework for carrying out an assessment of the impact an online resource is having.

As the toolkit web site emphasises:

There are a number of challenges in assessing the use and impact of online digital resources: these include new methods, shifts in the way that people access resources, new audiences, and new forms of information-seeking behaviour among different audiences.

The evaluation of online scholarship is a moving target, and therefore a flexible set of measures and practices will be used. The toolkit consists not of a single software solution, but a set of recommendations for best practices.

The toolkit inlcudes useful information, related articles, tools and guidance on how to use a range of quantitative and qualitatives measures (including webometrics, analytics, content analysis of media coverage, focus groups, resource surveys, user feedback analysis and more) and is open for submission of additional relevant resources, or comments, by the community.

The toolkit was piloted through case studies of five diverse JISC Digitisation projects funded under Phase 1 of the Digitisation programme.

In a previous post earlier in the year, Measuring the impact of digitised resources (12/6/2008), we announced the work that the Oxford Internet Institute (OII) was about to embark upon of identifying use and usage patterns of five JISC-funded online resources and devising meaningful metrics for the measurement of the impact of digitised scholarly resources.

This is a crucial tool for those interested in digitisation, providing much needed evidence and analysis of how digital resources are actually making a difference

At a recent JISC Digitisation programme meeting, Eric T Meyer and Katherine Eccles provided some background on how the OII is planning to carry out the work and the mixture of quantitative and qualitative measures they will take into consideration to gain some understanding of the use of such resources.

Quantitative Measures include:
• Webometrics
• Analytics
• Log file analysis
• Scientometrics / bibliometrics
• Content analysis of media coverage

Qualitative Measures include:
• Stakeholder interviews
• Resource surveys
• User feedback analysis
• Focus groups
• Questionnaires

One of the interesting things that emerged from their presentation was the need not to “obsess” too much about any particular indicator at any given time (eg, no need to look at web stats every month), but to consider a range of indicators collectively at regular intervals in time, in order to identify patterns over a longer duration of time.

The work will culminate in the creation of a Toolkit for the the Impact of Digitised Scholarly Resources, which will be disseminated in Spring 2009.

Presentation (PDF) on the Toolkit for the Impact of Digitised Scholarly Resources.

A recent report from OCLC on The Impact of Digitizing Special Collections on Teaching and Scholarship. Reflections on a Symposium about Digitization and Humanities highlights the main recommendations that emerged from the symposium held in June 2008.

The symposium brought together both primary users of (digitised) primary sources as well as “custodians”, such as librarians, archivists, museum professionals and senior managers. Participants discussed, from their own particular view points, strategies to maximise the impact of digitisation of special collections on teaching and research.

The report calls for specific directions for libraries and archives to take in the near future:

- work with faculty to understand current research methods and materials
- go outside the library or archive to build collections and work with faculty
- continue to build digital and material collections for both teaching and research.

Other important issues that emerged were:

- licencing and third-party agreements: the need for common principles in negotiating licensing contracts in order to ultimately guarantee open access to content
- Metrics: the need for more evidence of the impact of digitization and the acknowledgement that quantitative web stats on their own are just not enough.

On this last point: JISC is currently conducting an Impact study of the projects funded under Phase 1 of its Digitisation Programme. The project, carried out by the Oxford Internet Institute, will have as one of its key outcome the production of a Toolkit for the Impact of Digitised Resources, which will provide a framework for useful metrics to consider when assessing impact – see blog post on Measuring the impact of digitised resources.

The Stormont Papers resources makes available the debates from the parliament of Northern Ireland (Stormont) from creation in 1921 until the end of Home Rule in 1971.

It’s been available since 2006 and some statistics from the website are available. Of most interest is graph showing the spread of search terms entered by users

There are two interesting points from this

1) The bulk of your users may not be looking for the things you expect them to be looking for.

2) Pre-arranged hyperlinks on your home page can provide a user-friendly way of letting users get to know a resource’s contents.

Measuring the use and impact of digitised resources is no easy exercise. This is not only because of the changing nature of information seeking behaviour of different audiences, which has an effect on how users engage with digital resources. It is also due to the challenge in establishing appropriate metrics and criteria for measuring the impact and use of digitised collections.

When can a digital resource be considered a well-used resource? When and how do we know a resource has had an impact on its target audience? Are visitors’ numbers a useful performance indicator for judging the success of an online scholarly resource or should we dwell more on investigating what visitors actually do on a web site?

Yet, measuring impact is an important step in ascertaining if we’re reaching our audiences and responding to their needs. Impact has also been recognised as “the key factor in the potential for achieving long-term sustainability” for online academic resources in a recent report on Sustainability and Revenue Models for Online Academic Resources – an Ithaka Report commissioned by the Strategic Content Alliance.

Such are the issues, among many others, that the Oxford Internet Institute (OII) will be grappling with over the next year by leading on the JISC study on “Usage and Impact of Digitised Resources Funded Under the JISC Phase One Digitisation Programme”.

In addition to focusing on the assessment of five online collections created as part of the JISC Phase One Digitisation Programme, the OII will also produce a Toolkit for the Impact of Digitised Resources which aims to provide a set of approaches and tools available to measure and potentially improve the impact of current and future digitisation projects.

The Toolkit will be of value not only to JISC within the context of its digitisation programme, but also to the wider community. The OII study is due to terminate in Spring 2009 and both the final report and Toolkit will be made publicly available by JISC.

The JISC invites tenders to conduct a study on the usage and impact of a selection of online digital resources which were produced as part of the JISC Phase One Digitisation Programme, 2003-2007.

The digital collections created as part of the programme are aimed at enhancing the provision of e-content for teaching, learning and research purposes primarily in UK Higher and Further Education Institutions and to respond to the specific needs of users within this sector.

This study intends to investigate the level of impact, usage and take-up that such resources have had on teaching, learning and research within relevant subject areas and the degree to which they respond to users’ needs.

Funding of between £40,000 and £50,000, inclusive of VAT and expenses, is available for this study.

The deadline for proposals is 12 noon on Monday 28 April 2008. The study is expected to begin in June 2008 and be completed by March 2009.

Invitation To Tender document.