Leveraging Citation Management Software (CMS) for Broad Library Impact

Librarians recognized an opportunity to grow the number of RefWorks users on campus and launched an initiative to more aggressively market the software. Barriers to growth were recognized, staff training was undertaken, and targeted services and programming were infused with more content centered on the citation management software (CMS). After a year and a half, the library assessed the popularity and impact of the program. While RefWorks is now much more widely used on campus, it has also led to additional instructional classes, workshops, and reference transactions. These additional and unanticipated benefits have greatly enhanced the reputation of the software among librarians. In addition, students, faculty, and university administrators have all responded positively to the results of the initiative, enhancing the reputation of the library on campus. Other academic libraries may benefit from a similar campaign based on their own subscription CMS. became central to our broader mission. It also suggests how academic libraries can utilize a single research tool such as RefWorks to achieve various positive, and often unanticipated, results.


Introduction
Academic librarians know citation management software (CMS) is essential for conducting efficient scholarly research. Indeed, academic libraries typically provide access to and instruction for such tools to support the research activities of students and faculty (McMinn, 2011). Recently, the Franco Library at Alvernia University discovered that RefWorks, the CMS we have subscribed to for the last six years, can serve another vital purpose-to generally drive traffic to the reference desk, formal information literacy courses, workshops, and more. In fact, RefWorks has become a popular element across many of our services and much of our programming. While we once passively supported and occasionally mentioned RefWorks, we now leverage this valuable toolkit on a regular basis to build the library's reputation and value on campus.
This article reviews how we grew our base of RefWorks users beginning in January 2017, and furthermore how the CMS became central to our broader mission. It also suggests how academic libraries can utilize a single research tool such as RefWorks to achieve various positive, and often unanticipated, results.
Efforts to more fully integrate RefWorks arose from a broader review of library services. After assessing a wide range of data, we recognized that the provision of several vital services was on the decline. Most troubling of all, librarians were now teaching far fewer courses. In-person and online reference transactions, as well as consultations lasting more than twenty minutes, had all dropped substantially in recent years. The elimination of required library sessions for all First Year Seminar (FYS) courses following the 2015-16 academic year, the recent retirement of some long-serving librarians, and the retrenchment of one librarian position were the most obvious reasons for these disappointing trends. At a crossroads, librarians recognized the need to begin building one-on-one relationships with students and faculty members; additionally, we revised our model for recording reference transactions in the hopes of identifying opportunities for outreach. With our sensitivity heightened, we listened closely to what students were telling us. Before long, it was obvious that few of our returning students had ever used citation management software; even fewer had heard of RefWorks or knew that we provided access and support for this powerful tool. As we introduced these and other undergraduate and graduate students to RefWorks, most lamented the fact that they had not been introduced to the program sooner.
Though we had usage numbers on RefWorks, we had little to relate them to. At first glance, we might have assumed there was little room for growth. Perhaps we were already getting the most use we could reasonably expect from a tool we had already provided and promoted, however modestly, for more than four years. The rise in anecdotal evidence, however, suggested the rapidly changing landscape of library research had caught many returning students off guard. Graduate students returning to academia after many years are especially susceptible to this knowledge gap.
Indeed, Covert-Vail and Collard (2012) report that librarians recognize an essential need to provide graduate students with an "orientation towards core research behaviors" such as "basic proficiency with bibliographic management software such as . . . RefWorks" (p. 8). To accomplish this, we embraced the notion expressed by Buehler and Zald (2013) that to adequately "impact on graduate education, librarians need alternatives to the curricular integration strategy that has been so powerful for undergraduate information literacy efforts" (p. 219). Furthermore, the experience of Alvernia's librarians suggested many of the same skills were lacking among traditional undergraduates who often received only a one-shot freshmen year library session during their four years on campus. We sensed that many students would benefit from using RefWorks, if only they received an appropriate and timely introduction to the program. The cost of the CMS is not negligible and librarians were eager to learn if they were actually getting all they could for the money.
To better reach all of our students, we focused on a comprehensive approach to library services extending beyond the traditional bibliographic or information literacy class model employed in the past. RefWorks would play a major role in our strategy. A unique library tool, RefWorks is practical for all disciplines and for all campus populations.
We believed it fit well across a wide range of library services.

Review of Literature
For decades, researchers have considered the benefits of using CMS (Basak, 2014, Garfield;Flanagan, & Fox 1989). Many articles provide direct comparisons between specific tools (Basak, 2014;Gilmour & Cobus-Kuo, 2011;Hensley, 2011;Steeleworthy & Dewan, 2013); others outline criteria for selecting the best software for a particular academic library (Butros & Taylor, 2011;Marino, 2012). Tramullas, Sánchez-Casabón, and Garrido-Picazo (2015) have conducted a literature review of such studies. Researchers have also described advanced and less obvious benefits of CMS, such as its ability to encode research during the systematic review process (King, Hooper, & Wood, 2011) or to efficiently allow scholars to prepare manuscripts according to various journal styles (Steele, 2008). McMinn (2011) showed that roughly three-quarters of all academic libraries surveyed provide instruction in at least one of these tools. Though researchers have also articulated guidelines or recommended practices for library instruction or support of citation management software (Childress, 2011;Hensley, 2011), studies continue to show that these tools are underutilized by various campus populations. Salem and Fehrmann (2013) identified a tendency among undergraduates to ignore or abandon RefWorks because the return on their investment to learn it was insufficient.
Several studies indicate that only a minority of graduate students take advantage of CMS when conducting research (Gessner, Jaggars, Rutner, & Tancheva, 2011;Melles & Unsworth, 2015;Vezzosi, 2009;Wu & Chen, 2012). While a survey conducted at Manchester Metropolitan University by Harrison, Summerton, & Peters (2005) revealed that faculty desire more citation management software instruction for both themselves and undergraduate students, Niu et al. (2010) surveyed more than 2,000 academics in the U. S. and found that only a minority of them actually employ such citation databases. Several studies have reported on the specific ways academics employ CMS. Lorenzetti and Ghali (2013), for example, measured researchers' behavior when using reference software for systematic reviews and metaanalyses.
Studies have consistently demonstrated a need for robust instruction and support of CMS. Hensley (2011) argues that providing reference and instructional support for CMS in academic libraries "frees students to focus less on the minutia of generating citations and more on scholarly content" and permits librarians "to spend less energy teaching the specifics of styles and more time on not only why it is essential to properly cite but to introduce more advanced information management skills" (p. 208). Bussell, Hagman, and Guder (2017) believe that such tools provide logical content for library workshops. The same researchers, however, reinforce the findings of various studies when they conclude that "not all students consider learning such a tool to be a priority" (p. 980). The unique needs and habits of individual researchers have led to calls for a more comprehensive approach to teaching and supporting reference management (Emanuel, 2013;Melles & Unsworth, 2015).
Despite a rather rich history of research on CMS in academic libraries, little has been said about the potential to grow library services through CMS-centered programming. Several articles do suggest, if often only in passing, that library instruction in CMS can lead to more involvement with faculty on campus (Duong, 2010;East, 2001;Harrison, Summerton, & Peters, 2005;Hristova, 2012;Siegler & Simboli, 2002). Despite their vague promises of improved collaboration, however, these studies do not detail any wider impact on libraries which feature CMS in their programming.

Committing to RefWorks
Why were so few students and faculty members taking advantage of RefWorks? One reason, we concluded, was that our own information literacy curriculum was passing too quickly over the program in an effort to cover content considered more essential to a one-shot lesson, especially with freshman composition students. This specific gap in instruction was also consistent with a broader lack of the library's own proficiency and promotion of the CMS.

palrap.org
After acknowledging the expressed needs of students to learn RefWorks, and determining that both our undergraduate and graduate students would benefit from more robust training on the program, we also became aware that many faculty members were either unaware or unappreciative of RefWorks. If few professors recognized the program's value for their own research, even fewer realized how it could benefit their students' work. Childress (2011), however, cautions that "libraries should first consider the broader issue surrounding their role in supporting a more systematic approach to citation management as part of a comprehensive information literacy program" (p. 143). After much discussion, we concluded that the additional time and effort required of librarians to successfully support an ambitious new commitment to RefWorks was well worth it. Furthermore, we decided that any half measures would be inadequate if we were really going to accomplish our goal. The benefit to individual users, whatever their number might turn out to be, would be a good return on our investment.
Perhaps more importantly, we sensed that if we explained the benefits of RefWorks in the right way, we could use the program to build interest and enthusiasm for various other library services. Our only explicit goal was to grow the number of student and faculty RefWorks users on campus. The various services we introduced or infused with a dose of the CMS were meant to be vehicles toward that end. Much to our delight, RefWorks proved a symbiotic partner in such programming.
Though we were not new subscribers to RefWorks, our recommitment to the program led us to start from the beginning. In rough outline, we followed the best practices for integrating citation management software described by Hensley (2011), who among other steps called on libraries to "teach open workshops . . . integrate citation management into course-related instruction . . . look for opportunities to present at campus forums . . . conduct internal staff training . . . [and] let usage statistics guide [the] decision-making process" (p. 208). We followed each of these as we integrated RefWorks more consistently into our reference and instruction models. In a modified order, the elements of this overall strategy and the results we witnessed are explained in the sections that follow.

Following the Data
To identify trends and establish a baseline before launching our initiative, we collected and interpreted RefWorks data from July 2012, when we first subscribed to the program, through December 2016, when we determined to launch our plan. Figure 1 illustrates the number of what RefWorks considers "current users" during the first four years of our subscription. These data provide a benchmark measure after exposing the program to four consecutive freshmen classes and graduate cohorts. By the end of 2016, RefWorks growth was stagnant at best. Perhaps these numbers were the best we could hope for. Perhaps the decline in instructional classes, discussed below, or the growing popularity of products like Mendeley or Zotero, meant that RefWorks use had already peaked. Based on our full-time enrollment numbers, however, we knew there were many potential new RefWorks users we might still enlist. We also knew that we had never wholeheartedly promoted, instructed, or developed services emphasizing the benefits of the CMS. Unsure of how successful a new, concerted effort might be, we were convinced RefWorks deserved at least an earnest effort.

Building Staff Expertise
To properly launch an initiative promoting RefWorks, librarians would have to feel comfortable answering reference questions for what we anticipated to be a growing number of students and faculty members. An honest discussion among the library staff revealed we had only a basic familiarity with the program ourselves. Typically, we were pointing students toward our RefWorks LibGuide and other online resources rather than providing direct instruction or answers to questions about the program's many functions. This lack of confidence and familiarity largely stemmed from the fact that few librarians maintained an active RefWorks account and only a couple regularly used the program for their own citation management. This lack of experience and comfort is consistent with that of many academic librarians (Childress, 2011). But, as East (2001) points out, "building up a nucleus of staff who are competent users" of citation management software and "confident in their ability to instruct others in its use is not an easy task" (p. 69).
To grow our working knowledge of RefWorks, the Reference and Instruction Librarian led a December 2016 RefWorks refresher for all professional librarians. The session reviewed all RefWorks functions, but also included many tips for teaching the program to students and faculty. Aside from offering ideas on formal instruction, the workshop aimed to prepare all librarians for reference questions they might encounter when at the reference desk. During the session, particular scenarios in which RefWorks might be used to save time or troubleshoot research management issues were presented. Librarians had a chance to anticipate challenging questions and identify areas of weakness.
Librarians also created or began updating their existing accounts in an effort to build competency and comfort before turning toward instruction and reference.

Promoting RefWorks on Campus
While building our own expertise as users and teachers of RefWorks, we also began promoting the tool across campus. Beginning in January 2017, the Franco Library employed several strategies to increase awareness and use of RefWorks. We highlighted the CMS in our pre-semester instructional invitation email. The library's Spring 2017 newsletter featured an article on the value of RefWorks as well as a piece on our inaugural series of workshops, which would prominently feature the software.
During a departmental meeting with the provost, we also championed the new steps we planned to take.
Quite unexpectedly, she invited a librarian to address the entire faculty. In response, the Reference and Instruction Librarian prepared and delivered a presentation to the full faculty during a January 2017 in-service meeting. The presentation stressed the value of the citation manager for professors frustrated with inadequate student attention to formal style requirements. While promoting RefWorks for all students, the presentation implied that knowledge and use of such tools could greatly benefit anyone undertaking graduate level study in today's digital information landscape. Immediately following the presentation, several faculty members requested help initiating their own palrap.org accounts. Among the requests for information literacy sessions in the days that followed, we were asked many times to specifically include a segment on RefWorks.

Integrating RefWorks into Information Literacy Classes
Beginning in January of 2017, we determined RefWorks should play a more integral part in our instructional services. Lessons once focused almost exclusively on surveying library resources and teaching specific search strategies now show students how to create a RefWorks account and often also include a brief tutorial during which students practice importing and organizing database sources. By devoting roughly 20% of IL instructional time to RefWorks, the program can supplement the entire lesson, typically permitting students to upload to RefWorks relevant sources discovered by means of the search techniques we introduce throughout the class period. This conscious layering of practice reinforces the notion of RefWorks as a foundational tool. Librarians focus on the metanarrative of student research by stressing the importance of recognizing one's own organizational practices. Thus, our focus on RefWorks required a pedagogical shift from almost exclusively emphasizing the discovery of resources to instead embedding research strategies within a larger sense of research workflow awareness and improvement.
While teaching, we distinguish RefWorks from tools such as EasyBib, NoodleBib, or others with which students may already be familiar and comfortable. We demonstrate the additional benefit of long-term research management and full integration with the paper writing process available only from a subscription-based tool like RefWorks.
For honors students, seniors completing a capstone assignment, and graduate students, we illustrate ways to employ RefWorks during the literature review process. We show these students how to annotate and add tags to references in order to create subcategories within a larger research collection. The benefits of such advanced features are well documented (King, Hooper, & Wood, 2011;Steele, 2008), yet they are not easily discoverable by students, who nevertheless report to us that they value learning about them in our class sessions. When time does not permit such an introduction, librarians suggest that professors have us return for a short segment devoted to RefWorks. During formal instruction, we additionally market our new workshop on RefWorks, as well as individual consultations, as opportunities to learn more about the program.
We also openly discuss the shortcomings of RefWorks. Librarians emphasize that students should expect the program to sometimes generate faulty references. We caution students that references will only be as good as the records they manually enter or import from databases. Warning students to beware the shortcomings of RefWorks also helps preempt complaints that may arise later. Librarians build trust when they admit their own occasional frustrations with the program. Our main message is that the CMS is a vast improvement over alternative non-subscription tools as well as independent strategies for managing research.

Launching a RefWorks-centered Workshop Series
With RefWorks prominently featured within our broader educational objectives, the software quite naturally served as the centerpiece for a new workshop series.
Despite research indicating many graduate students prefer to acquire new skills through online tutorials (Fong, Wang, White, & Tipton, 2016), we decided to concentrate on classroom-based instructional approaches. By prioritizing face-to-face interactions, we could simultaneously build the rapport we hoped would translate to subsequent interactions in the classroom and at the reference desk. This also proved practical as we chose to expand our audience beyond graduate students. To better serve multiple campus communities and to provide an opportunity for enrollees to advance through a series of workshops, we offered a wide menu of topics. The value of multiple palrap.org workshops is supported by Wong and Cmor (2011), who found that student GPAs are more likely to rise when students enrolled in multiple sessions.
Three of our 2017-18 workshops, "RefWorks: Your Personal Research Manager," "Library Research for the Graduate Student," and "Search Strategies for the Literature Review," prominently featured RefWorks. Because the software was strongly endorsed and briefly described during the "Library Research" and "Search Strategies" workshops, several attendees of these sessions subsequently registered for the "RefWorks" workshop.

Results
The preceding sections illustrate how we have enhanced the presence of RefWorks across our library's services. These efforts primarily yielded an increased number of new RefWorks accounts (Figure 2). During the first three months of our initiative (January-March 2017) we added 202 new accounts, a total 29 times greater than we witnessed in the same three months of 2015, for example, when growth was stagnant with only 7 new accounts created.
A more consistent instructional approach to RefWorks likely accounts for many of these additional registrations. More assuring than new or existing account growth, however, is the steady rise in the number of references created (see   As we increased programming for RefWorks, we expected it would be most enthusiastically received by graduate students. We discovered, however, that it was also relatively easy to grow the number of undergraduates who use the program. This improvement is consistent with the findings of Kiernan (2006), who found that fully half of RefWorks users at the University of Minnesota were undergraduates. Between December 2016 and June 2018, we more than doubled the number of undergraduates with RefWorks accounts, outpacing the impressive 83% growth in graduate student accounts. The 113% gain in current accounts among faculty members, however, was the biggest surprise of all.
Though facility with RefWorks still varies from librarian to librarian, each member of the reference team is now closer to the ideal of functional awareness and familiarity described by Childress (2011). During the last year, all librarians have successfully fielded many questions related to the program. The rising number of reference questions has also contributed to our general preparedness and confidence.
Librarians now routinely tout RefWorks during reference transactions and during both casual discussions and instructional planning meetings with faculty members. More than a year into our initiative, students and faculty are now much more likely to acknowledge knowing what the program is and does.

Figure 4 Number of Library Classes Taught by Academic Year *Last Year Library Instruction Required for all FYS Courses
The successful launch of the library's "Savvy Scholar" workshop series also owes much to RefWorks. When the fall series of workshops ended, the session on RefWorks proved the most popular (see Table 1). Of the 89 total workshop seats filled during the fall of 2017, more than a third were in the RefWorks sessions.

Discussion
While we cannot be certain that RefWorks caused the number of instructional sessions to increase or that it led students and faculty to sign up for workshops on other topics, its growth nevertheless correlates with that seen in these additional services. Furthermore, RefWorks played a central part in both of these library services during the last year and a half. Of the five workshop topics we offered, RefWorks was the most popular, indicating the program's inherent appeal. Furthermore, the workshop's popularity suggests that even those students who heard about RefWorks outside of formal classes were interested in learning more about it. Additionally, the growth in formal facultyrequested instructional classes included several specific requests for RefWorks-only lessons.
It is unlikely that the growth in RefWorks users and instructional classes were the products of growth in overall library use. While reference transactions related to RefWorks have risen since we began embracing the program, overall reference interactions have declined. Furthermore, physical traffic through the library has been declining, while our LibGuides visits have remained relatively steady. Essentially, RefWorks has positively impacted our services at a time when little growth seemed likely.
Students and faculty members now routinely share positive, and even enthusiastic, reports of their experience using the CMS. While only anecdotal, such unsolicited endorsements signal to us a new popularity and genuinely positive attitude toward RefWorks.
While adopting a systematic approach to building the RefWorks community of users on campus, we also gained a serendipitous momentum. The presence of the CMS in our daily routines expanded by natural and unanticipated means. Similar experiences were reported by Turner (2017) and are perhaps best encapsulated by Cranston and Level (2006), who explain how from "a faculty workshop, a whole series of instruction sessions blossomed…proving even small steps can yield great strides" (pp. 20-21). Indeed, such tangential growth in library services, specifically reference transactions, as a result of bibliographic instruction has been quantified by Saunders (2003).
Best of all, librarians talk more about RefWorks. In our staff meetings and informal discussions, the management tool figures prominently when we discuss reference and instruction. Librarians working with students and faculty at the reference desk consistently and confidently recommend RefWorks in relation to specific research projects and broader strategies shared with students. In general, we now situate RefWorks within a larger context of tools and practices from which individual students and researchers may assemble personalized research management routines. Demonstrating how RefWorks fits into a scholarly routine is consistent with research dispositions described in the new ACRL framework. In general, we see affirmation that such tools "can be effectively incorporated into an individual's daily workflow" (Owen, 1997, p. 30).
When we began our initiative, we suspected students might not appreciate the value of the program. We assumed they might be reluctant to take the advice of librarians and break their existing research habits. Ironically, students proved the easiest to convince. Overcoming the trepidation of librarians and faculty members was more challenging. In general, however, there has been large buy-in for a program that had been hidden in plain sight on our campus for years.

Future Considerations
The increase in graduate classes visiting the library for information literacy instruction combined with the successful launch of workshops are encouraging signs. The efforts to build stronger relationships with Alvernia's graduate programs has paid off. There is, however, more opportunity to expand the number of students we reach.

palrap.org
Ultimately, a for-credit course in graduate level library research would ensure that all enrolled students receive a thorough working knowledge of advanced research strategies, including various CMS options. The positive response to current workshop offerings suggest that adequate demand already exists for such a course.
Because research points to the benefits of targeting sophomores and upper division students for whom foundational research skills alone eventually prove insufficient (Black, 2014;Bowles-Terry, 2012;D'Couto & Rosenhan, 2015;Kuglitsch & Burge, 2016), we hope to also employ RefWorks as a way to reach more undergraduates following their freshmen year.
Following the call of Lonergan (2017), we plan to survey faculty and students about their use of RefWorks as well as other CMS. Learning more about the most popular features of these programs can aid us in future marketing efforts and in designing instruction to address knowledge gaps among current users. Such research should reveal idiosyncrasies among RefWorks users like those identified by various studies (Hristova, 2012;Lonergan, 2017;Melles & Unsworth, 2015).
In general, we can now tout the success we had with RefWorks when pitching future initiatives among librarians and campus administrators. Our current provost recently praised us after we shared with him much of the data contained in this article. Our CMS success story shows above all that we are working to help students. It has enhanced our reputation among students and faculty, while instilling a sense of pride and achievement among our librarians.

Conclusion
Citation management software can do much more than manage citations and improve the research workflows of students and faculty. Ultimately, the software gives librarians an opportunity to offer very practical advice to a larger number of library users. Programs such as RefWorks provide tangential benefits when featured widely across library services and programs. Academic libraries are encouraged to reconsider traditional attitudes toward CMS and devote additional resources to promoting such tools-ultimately, students and faculty will greatly benefit when they do.