GLAM/Evaluation/ar

This page is a translated version of the page GLAM/Evaluation and the translation is 1% complete.

Over the past years, the GLAM-WIKI-community has collected a tremendous amount of experience with GLAM-Outreach activities and GLAM-WIKI cooperations. So far, there have however been only a few systematic approaches to the evaluation of such cooperations. In order to reach a shared vision of what may be achieved by different types of GLAM outreach activities, past case study reports have been analyzed and a workshop was carried out at Wikimania 2013.

GLAM

Galleries • Libraries • Archives • Museums

البدء • مشاريع نموذجية ودراسات حالة • تقييم المشاريع • الاتصال بنا
لمجتمع جلام الويكي: تواصل • النقاش • تقويم • النشرات الإخبارية • موارد • متطوعون • صفحات أخرى

The results of the analysis and the workshop are summarized below, organized according to different types of GLAM-WIKI cooperations. They can serve as a reference both during the planning stage and for the evaluation of a cooperation.

This overview page, and especially also the lists of indicators and measurement methods, are work-in-progress and will be complemented based on further experiences. A first step in this regard has already been made by integrating some of the insights derived from the program evaluation of edit-a-thons.

General outcomes Volunteer Photography Batch Uploads and content donations Digitization Wikipedia editing Wiki Loves and Wiki Takes events Embedding Wikimedia Content in Exhibits and Institutional Websites Embedding Wikimedians in Institutions Embedding Wikimedians in Institutions Developing volunteer communities Building institutional capacity

Regarding GLAM-WIKI cooperations in general

Indicators of success / metrics

The following indicators of success should be considered regardless of the type of activity:

Indicators related to participation (how well is the cooperation supported by the community?)
Indicators related to multiplication (if an activity is successful, you may want to think about how to make it last and/or how to extend it to other areas or other partner institutions)
Indicators related to public relations (should be considered if there is anything interesting to tell about the cooperation)
Indicators related to resource spending (should be considered if you are interested in evaluating the cost-effectiveness of an activity)

التحديات

Offline activity can lead to a reduction of online activity and in some cases even cause burnout (resulting in people leaving the project)
Measuring participants' on-wiki activities is difficult and poses ethical challenges
GLAM-WIKI cooperations often suffer from insufficient documentation and follow-up

Best practices

Get "peers" (i.e. other GLAMs) to promote GLAM-WIKI cooperations
Attract additional volunteers for offline activities
Use Wikimetrics to measure participants' on-wiki activities
Devote staff time to organisational learning

Facilitating Discovery / Facilitating Photographs

Examples: Visits, guided tours, back stage passes

Positive outputs / outcomes

Pictures of the institution's collection were uploaded to Wikimedia Commons
Wikipedians got to know the internal processes of the GLAM (e.g. archival process)
Wikipedians learned about new topics to write about
Wikipedians got access to closed collections and archives

Indicators of success / metrics

Typical follow-up activities in cooperation with an institution / can be combined with

Cooperation talk
edit-a-thon
Writing competition

Content Donations / Content Partnerships

Examples: image / multimedia donations; text donations (e.g. GLAMs freely licensing their texts to contribute to articles); document donations (in view of their use on Wikisource)

Positive outputs / outcomes

Additional images and/or multimedia files uploaded to Wikimedia Commons
Images / multimedia files embedded in Wikipedia articles
Free licensing of description texts and background information on the partner GLAM's website
Free licensing of images on the partner GLAM's website and database
Increased web-traffic on the partner GLAM's website
Improvement or correction of the partner GLAM's metadata thanks to community feedback
Improvement or correction of Wikimedia content/data thanks to input from the GLAM

Indicators of success / metrics

Digitization

Examples: Joint digitization project in partnership with a GLAM

Positive outputs / outcomes

Images, document scans, or media files uploaded to Commons

Indicators of success / metrics

Indicators related to pictures / media files

Typical challenges

Audio / video digitization
Taking a decision with regard to professional vs. do-it-yourself digitization
Financial restrictions

Best practices

Create a knowledge base for do it yourself digitization
Set up / identify funding programs for digitization

Typical follow-up activities in cooperation with an institution / can be combined with

Backstage pass

Article Improvement

Examples: Writing challenges, Edit-a-thons

Positive outputs / outcomes

Improved Wikipedia articles (e.g. promoted articles to featured status)
Created new Wikipedia articles, especially in subjects that are otherwise poorly covered by Wikipedia
Created new Wikipedia articles in various languages related to the partner institution's collection
Created templates related to the partner institution or its collection
Acquired new contributors
Received media coverage
GLAM staff provided expertise and facilitated access to reference material
Wikipedians and GLAM staff got to know each other and each other's organisational culture

Indicators of success / metrics

Typical follow-up activities in cooperation with an institution / can be combined with

Backstage pass

Crowdsourcing the Collection of Photographs

Examples: competitions to collect pictures, "Wiki Takes ... Events"

Positive outputs / outcomes

Pictures uploaded to Commons
Acquired new contributors

Indicators of success / metrics

Embedding Wikimedia Content inside a GLAM

Examples: QRpedia, Wikipedia articles in exhibitions

Positive outputs / outcomes

Generated additional Wikipedia page views (especially also to articles about less known subjects)
Created new Wikipedia articles
Added images from the GLAM's collection to Commons
Got parents / older siblings to show/explain Wikipedia content to children

Indicators of success / metrics

Typical follow-up activities in cooperation with an institution / can be combined with

Article improvement through a writing challenge or edit-a-thon

Connecting GLAM collections to Wikimedia content

Examples: Improving articles about items in the GLAM's collection, referencing Wikipedia articles with GLAM content

Positive outputs / outcomes

Improved the article(s) about some of the items in the GLAM partner's collections more content, neutral tone, references; Good Article / Featured Article status
Familiarized GLAM staff with Wikipedia
Received input from GLAM staff concerning relevant content and sources in their area of expertise
Created articles about subjects that are underrepresented on Wikipedia due to the institutional bias of the community

Indicators of success / metrics

Developing editor communities in Wikimedia projects

Involvement in GLAM collaborations turns a wikimedians hobby on their computer into something that commands respect from family and friends who may not respect an internet hobby. Backstage passes etc are an opportunity for longstanding wikipedians to bring their other halves. Events in a particular location or subject area are an opportunity to remotivate inactive wikimedians who are in that area or interested in that subject.

All this contributes to enabling GLAM outreach to help tackle Wikimedia's longstanding problem with editor retention and develop communities of practice that are interested in supporting that knowledge.

Examples: Liam Wyatt organised the first GLAM editathon in June 2010. Eight Wikimedians signed up to join Liam and the curators at the venue. 37 months later all 9 of those Wikimedians are still active in the community.

Positive outputs / outcomes

Wikimedians who participate in GLAM activities are likely to remain active in the Wikimedia community for longer than similar editors
Wikimedians who are invited to participate in GLAM activities are likely to remain active in the Wikimedia community for longer than similar editors
Museum collaborations are an opportunity to reach out to relevant but inactive Wikimedians

Indicators of success / metrics

Indicators related to contributors

Combining GLAM-Outreach and Education

Examples: Student programs in connection with GLAMs

Positive outputs / outcomes

Created new articles about the GLAM's holdings
Uploaded images to Wikimedia Commons
Recruited new contributors

Indicators of success / metrics

Typical challenges

Students take more time than expected
Only few students continue contributing
Wiki-markup is a challenge
Find new topics every year
Copyright issues
Terminology clashes between the teachers / academic community and the Wikipedia community

Best practices

Reach out to students' interest group in order to enhance retention rates
Start with an edit-a-thon to overcome problems with wiki-markup
Use peer-review between students
Use a bot to report students' activities

Combining GLAM-Outreach and Professional Training

Examples: (Social media) internship at a GLAM including work on Wikipedia articles

Positive outputs / outcomes

Created / improved Wikipedia articles about the GLAM's collection
Familiarized a future GLAM professional with Wikipedia procedures and values

Indicators of success / metrics

Indicators related to article creation and improvement

Embedded Wikimedians

Examples: Wikipedian in Residence

Positive outputs / outcomes

GLAM staff learned about Wikipedia/Wikimedia practices and values
GLAM staff learned about free licensing
Created / improved articles about the institution's collection
Uploaded images to Commons
Acquired new contributors
Presented Wikipedia and its sister projects at events organized by the GLAM partner or by the respective professional community
Received media coverage
Put GLAM staff and interested Wikipedians in touch / created new relationships
Created a direct communication channel (contact page) between the GLAM partner and the Wikipedia community
Museums: Attracted more visitors to the museum

Indicators of success / metrics

Typical challenges

Differing expectations according to the type of GLAM (e.g. museums are mostly interested in attracting more visitors)
(Paid) residencies are rather short
Africa: Reliance on remote communication (in case of Africa projects coordinated/supported from Europe)

Best practices

The range of potential activities is very large and time is usually limited. Therefore: agree on project goals in advance and secure top management support.
Focus on training and institutionalization of the cooperation (make sure that the work is being continued; activate the GLAM's community, such as friends of the GLAM or academics in related fields)

Typical follow-up activities in cooperation with an institution / can be combined with

Content improvement in partnership with curators and Wikipedians (edit-a-thons)
Content improvement through writing challenges / competitions
Image and/or multimedia donations
Visits, backstage passes
QRpedia codes within the GLAM
Students' activities in connection with the GLAM

Capacity Building

Examples: Wikipedia/Wikimedia training for GLAM staff and volunteers; GLAM-Wiki conferences directed at professionals; GLAMCamps to foster the exchange / improve the coordination among Wikimedians

Positive outputs / outcomes

Informed GLAM representatives about possibilities to cooperate with Wikipedia/Wikimedia

Indicators of success / metrics

Indicators related to multiplication

Typical follow-up activities in cooperation with an institution

Content partnerships
Wikipedian in Residence / internship