GLAM/Evaluation/eo
Over the past years, the GLAM-WIKI-community has collected a tremendous amount of experience with GLAM-Outreach activities and GLAM-WIKI cooperations. So far, there have however been only a few systematic approaches to the evaluation of such cooperations. In order to reach a shared vision of what may be achieved by different types of GLAM outreach activities, past case study reports have been analyzed and a workshop was carried out at Wikimania 2013.
Galleries • Libraries • Archives • Museums
Komenciĝi • Modelaj Projektoj kaj Kazostudoj • Taksi Projektojn • Kontakti Nin
Por la Galeria-Biblioteka-Arkiva-Muzea Vikia Komunumo: Konekti • Diskuto • Kalendaro • Informilo • Rimedoj • Volontuloj • Aliaj paĝoj
The results of the analysis and the workshop are summarized below, organized according to different types of GLAM-WIKI cooperations. They can serve as a reference both during the planning stage and for the evaluation of a cooperation.
This overview page, and especially also the lists of indicators and measurement methods, are work-in-progress and will be complemented based on further experiences. A first step in this regard has already been made by integrating some of the insights derived from the program evaluation of edit-a-thons.
Regarding GLAM-WIKI cooperations in general |
Indicators of success / metrics
The following indicators of success should be considered regardless of the type of activity:
- Indicators related to participation (how well is the cooperation supported by the community?)
- Indicators related to multiplication (if an activity is successful, you may want to think about how to make it last and/or how to extend it to other areas or other partner institutions)
- Indicators related to public relations (should be considered if there is anything interesting to tell about the cooperation)
- Indicators related to resource spending (should be considered if you are interested in evaluating the cost-effectiveness of an activity)
Defioj
- Offline activity can lead to a reduction of online activity and in some cases even cause burnout (resulting in people leaving the project)
- Measuring participants' on-wiki activities is difficult and poses ethical challenges
- GLAM-WIKI cooperations often suffer from insufficient documentation and follow-up
Bonaj praktikoj
- Get "peers" (i.e. other GLAMs) to promote GLAM-WIKI cooperations
- Attract additional volunteers for offline activities
- Use Wikimetrics to measure participants' on-wiki activities
- Devote staff time to organisational learning
Facilitating Discovery / Facilitating Photographs |
Examples: Visits, guided tours, back stage passes
Positive outputs / outcomes
- Pictures of the institution's collection were uploaded to Wikimedia Commons
- Wikipedians got to know the internal processes of the GLAM (e.g. archival process)
- Wikipedians learned about new topics to write about
- Wikipedians got access to closed collections and archives
Indicators of success / metrics
Typical follow-up activities in cooperation with an institution / can be combined with
- Cooperation talk
- edit-a-thon
- Writing competition
Content Donations / Content Partnerships |
Examples: image / multimedia donations; text donations (e.g. GLAMs freely licensing their texts to contribute to articles); document donations (in view of their use on Wikisource)
Pozitivaj eligoj / rezultoj
- Additional images and/or multimedia files uploaded to Wikimedia Commons
- Images / multimedia files embedded in Wikipedia articles
- Free licensing of description texts and background information on the partner GLAM's website
- Free licensing of images on the partner GLAM's website and database
- Increased web-traffic on the partner GLAM's website
- Improvement or correction of the partner GLAM's metadata thanks to community feedback
- Improvement or correction of Wikimedia content/data thanks to input from the GLAM
Indicators of success / metrics
Bitigo |
Examples: Joint digitization project in partnership with a GLAM
Positive outputs / outcomes
- Images, document scans, or media files uploaded to Commons
Indicators of success / metrics
Typical challenges
- Audio / video digitization
- Taking a decision with regard to professional vs. do-it-yourself digitization
- Financial restrictions
Best practices
- Create a knowledge base for do it yourself digitization
- Set up / identify funding programs for digitization
Typical follow-up activities in cooperation with an institution / can be combined with
- Backstage pass
Article Improvement |
Examples: Writing challenges, Edit-a-thons
Pozitivaj eligoj / rezultoj
- Improved Wikipedia articles (e.g. promoted articles to featured status)
- Created new Wikipedia articles, especially in subjects that are otherwise poorly covered by Wikipedia
- Created new Wikipedia articles in various languages related to the partner institution's collection
- Created templates related to the partner institution or its collection
- Acquired new contributors
- Received media coverage
- GLAM staff provided expertise and facilitated access to reference material
- Wikipedians and GLAM staff got to know each other and each other's organisational culture
Indicators of success / metrics
- Indicators related to article creation and improvement
- Indicators related to contributors
- Indicators related to GLAM-WIKI interaction
- See also: Program Evaluation Page for Edit-a-thons
Typical follow-up activities in cooperation with an institution / can be combined with
- Backstage pass
Crowdsourcing the Collection of Photographs |
Examples: competitions to collect pictures, "Wiki Takes ... Events"
Pozitivaj eligoj / rezultoj
- Pictures uploaded to Commons
- Acquired new contributors
Indicators of success / metrics
Embedding Wikimedia Content inside a GLAM |
Examples: QRpedia, Wikipedia articles in exhibitions
Pozitivaj eligoj / rezultoj
- Generated additional Wikipedia page views (especially also to articles about less known subjects)
- Created new Wikipedia articles
- Added images from the GLAM's collection to Commons
- Got parents / older siblings to show/explain Wikipedia content to children
Indicators of success / metrics
- Indicators related to article creation and improvement
- Indicators related to pictures / media files
- Indicators related to visitors
Typical follow-up activities in cooperation with an institution / can be combined with
- Article improvement through a writing challenge or edit-a-thon
Connecting GLAM collections to Wikimedia content |
Examples: Improving articles about items in the GLAM's collection, referencing Wikipedia articles with GLAM content
Pozitivaj eligoj / rezultoj
- Improved the article(s) about some of the items in the GLAM partner's collections more content, neutral tone, references; Good Article / Featured Article status
- Familiarized GLAM staff with Wikipedia
- Received input from GLAM staff concerning relevant content and sources in their area of expertise
- Created articles about subjects that are underrepresented on Wikipedia due to the institutional bias of the community
Indicators of success / metrics
Developing editor communities in Wikimedia projects |
Involvement in GLAM collaborations turns a wikimedians hobby on their computer into something that commands respect from family and friends who may not respect an internet hobby. Backstage passes etc are an opportunity for longstanding wikipedians to bring their other halves. Events in a particular location or subject area are an opportunity to remotivate inactive wikimedians who are in that area or interested in that subject.
All this contributes to enabling GLAM outreach to help tackle Wikimedia's longstanding problem with editor retention and develop communities of practice that are interested in supporting that knowledge.
Examples: Liam Wyatt organised the first GLAM editathon in June 2010. Eight Wikimedians signed up to join Liam and the curators at the venue. 37 months later all 9 of those Wikimedians are still active in the community.
Pozitivaj eligoj / rezultoj
- Wikimedians who participate in GLAM activities are likely to remain active in the Wikimedia community for longer than similar editors
- Wikimedians who are invited to participate in GLAM activities are likely to remain active in the Wikimedia community for longer than similar editors
- Museum collaborations are an opportunity to reach out to relevant but inactive Wikimedians
Indicators of success / metrics
Combining GLAM-Outreach and Education |
Examples: Student programs in connection with GLAMs
Pozitivaj eligoj / rezultoj
- Created new articles about the GLAM's holdings
- Uploaded images to Wikimedia Commons
- Recruited new contributors
Indicators of success / metrics
- Indicators related to article creation and improvement
- Indicators related to pictures / media files
- Indicators related to contributors
Tipaj defioj
- Students take more time than expected
- Only few students continue contributing
- Wiki-markup is a challenge
- Find new topics every year
- Copyright issues
- Terminology clashes between the teachers / academic community and the Wikipedia community
Bonaj praktikoj
- Reach out to students' interest group in order to enhance retention rates
- Start with an edit-a-thon to overcome problems with wiki-markup
- Use peer-review between students
- Use a bot to report students' activities
Combining GLAM-Outreach and Professional Training |
Examples: (Social media) internship at a GLAM including work on Wikipedia articles
Pozitivaj eligoj / rezultoj
- Created / improved Wikipedia articles about the GLAM's collection
- Familiarized a future GLAM professional with Wikipedia procedures and values
Indicators of success / metrics
Embedded Wikimedians |
Examples: Wikipedian in Residence
Pozitivaj eligoj / rezultoj
- GLAM staff learned about Wikipedia/Wikimedia practices and values
- GLAM staff learned about free licensing
- Created / improved articles about the institution's collection
- Uploaded images to Commons
- Acquired new contributors
- Presented Wikipedia and its sister projects at events organized by the GLAM partner or by the respective professional community
- Received media coverage
- Put GLAM staff and interested Wikipedians in touch / created new relationships
- Created a direct communication channel (contact page) between the GLAM partner and the Wikipedia community
- Museums: Attracted more visitors to the museum
Indicators of success / metrics
- Indicators related to GLAM-WIKI interaction
- Indicators related to article creation and improvement
- Indicators related to pictures / media files
- Indicators related to contributors
- Indicators related to public relations
- Indicators related to visitors (in case of museums)
Typical challenges
- Differing expectations according to the type of GLAM (e.g. museums are mostly interested in attracting more visitors)
- (Paid) residencies are rather short
- Africa: Reliance on remote communication (in case of Africa projects coordinated/supported from Europe)
Bonaj praktikoj
- The range of potential activities is very large and time is usually limited. Therefore: agree on project goals in advance and secure top management support.
- Focus on training and institutionalization of the cooperation (make sure that the work is being continued; activate the GLAM's community, such as friends of the GLAM or academics in related fields)
Typical follow-up activities in cooperation with an institution / can be combined with
- Content improvement in partnership with curators and Wikipedians (edit-a-thons)
- Content improvement through writing challenges / competitions
- Image and/or multimedia donations
- Visits, backstage passes
- QRpedia codes within the GLAM
- Students' activities in connection with the GLAM
Capacity Building |
Examples: Wikipedia/Wikimedia training for GLAM staff and volunteers; GLAM-Wiki conferences directed at professionals; GLAMCamps to foster the exchange / improve the coordination among Wikimedians
Pozitivaj eligoj / rezultoj
- Informed GLAM representatives about possibilities to cooperate with Wikipedia/Wikimedia
Indicators of success / metrics
Typical follow-up activities in cooperation with an institution
- Content partnerships
- Wikipedian in Residence / internship