Web Presence and Higher Education Rankings
by Steve McCarty
Professor, Osaka Jogakuin College and Osaka Jogakuin University, Japan
Higher education rankings receive much attention from the mainstream mass media, educational administrators in a competitive economic environment, educators concerned with their careers, prospective students, and other stakeholders including the general public worldwide. This paper will first review salient aspects of prominent higher education ranking organizations, such as where to find them on the Web, their key characteristics, criteria for ranking, and where to find more information about their methodology. The workings of these organizations and the data that they analyze are on the Web to such an extent that ranking issues enter the purview of online education in a broad sense.
The increasing weight of Web presence in the criteria will be observed, suggestive of possible institutional adaptations or actions individuals can take. Leaving aside factors beyond the control of individuals, the ranking criteria clarify in some ways how institutions and faculty members will increasingly be measured and thereby evaluated. Educators with a stake in the rankings, who might have an effect on the results, may be concerned about how to raise their scholarly profile while serving institutional goals of achieving the highest possible ranking. This does not mean to game the system but rather to receive due recognition by matching the format of accomplishments with the measuring tools and criteria of the ranking organizations. This paper therefore examines ranking criteria in order to clarify the terrain objectively and suggest some Web-based means whereby individual educators can optimize their scholarly profile. The conclusion will focus on Google Scholar as an effective approach in view of the criteria, while future articles will be needed to provide more specific directions and suggestions among the many possibilities.
Higher Education Ranking Criteria
Name: Times Higher Education World University Rankings
Key characteristics: Ranks the top 400 universities in the world, by region, and by subject.
Criteria: “The performance indicators that form the basis of the rankings fall into five areas: teaching, research, citations, international outlook and interaction with industry” (“Asian universities,” 2012).
For further information on methodology: http://www.timeshighereducation.co.uk/world-university-rankings/2012-13/world-ranking/methodology
Name: SCImago Institutions Rankings
Key characteristics: “includes 3,290 institutions that together are responsible for more than 80% of worldwide scientific output” (quotations are from its Website). It has global and regional rankings, such as for Latin America and Asia. It is research-oriented and thus includes institutes and corporations in addition to universities. SCImago data are used by other ranking systems like Webometrics (below).
Criteria: “scientific output during the term 2006-10 as indexed in Elsevier’s Scopus database.”
For further information on methodology: http://www.scimagoir.com/methodology.php
Name: Webometrics – Ranking Web of World Universities
Key characteristics: It ranks the Web presence and performance of over 20,000 universities in the world. Size refers to the output of Web pages. It has rankings also for regions and individual countries. It claims that on-campus survey data would not be reliable, and that institutions will rapidly lose ground if they do not direct their efforts online.
Criteria: Having changed considerably since early 2012, the criteria are actively evolving, but still worth studying in detail at the site below. Currently Presence (20%) refers to Web pages in the university’s domain found by Google. They advise against using other domains, and they concede that large universities have an advantage. They recommend that all university staff contribute content pages to the university’s domain. Impact (50%) refers to the number of links and different institutions linking to the university’s Website, a form of referendum or review, but which is partly influenced by prestige or popularity, as measured by Majestic SEO and ahrefs. Openness (15%) refers to repositories and the number of rich files (pdf, doc, docx, ppt, pptx; apparently not html pages) of academic content over the past five years found on the university’s domain by the academic search engine Google Scholar. Excellence (15%) refers to the number of papers in the top 10% cited in each field from among major international journals in the last eight years, as measured by SCImago.
For further details and information on methodology: http://www.webometrics.info/en/Methodology
Name: 4 International Colleges & Universities
Key characteristics: “an international higher education directory and search engine featuring reviews and web rankings of 11,000 Universities and Colleges in 200 countries” (from its Website). This multi-purpose site includes global, regional, and national rankings, plus listings of universities on social media and other classifications.
Criteria: It requires institutions to be accredited, with at least four-year degree programs, and which provide traditional f2f learning facilities and courses. It utilizes Google Page Rank, Alexa Traffic Rank, and Majestic SEO Referring Domains.
For further details and information on methodology: http://www.4icu.org/menu/about.htm
Name: QS World University Rankings
Key characteristics: Part of a site to help international students choose suitable universities abroad. It ranks universities overall, by region, and according to each criterion such as citations per faculty.
Criteria: Academic Reputation (40%) and Employer Reputation (10%) are determined by global survey data. Citations per faculty member (20%) utilize data from Sciverse Scopus. They also search for data on faculty-student ratio (20%), proportion of international students (5%) and faculty members (5%).
For further information on methodology: http://www.iu.qs.com/university-rankings/world-university-rankings/
Name: U.S. News World’s Best Universities
Key characteristics: Based on data from the QS World University Rankings (above).
Name: Academic Ranking of World Universities
Key characteristics: Shanghai Jiao Tong University ranks the top 500 universities in the world, and by field and subject. The criteria are mainly measures of research performance.
Criteria: “ARWU uses six objective indicators to rank world universities, including the number of alumni and staff winning Nobel Prizes and Fields Medals, number of highly cited researchers selected by Thomson Scientific, number of articles published in journals of Nature and Science, number of articles indexed in Science Citation Index – Expanded and Social Sciences Citation Index, and per capita performance with respect to the size of an institution” (from its Website).
For further information on methodology: http://www.shanghairanking.com/aboutarwu.html
Key characteristics: The G-factor International University Ranking does not seem to have a site currently, but it has been used by other ranking systems and shows the growing importance of Web presence. It measures the number of links from leading international universities to a university’s Websites, utilizing the Google search engine. It emphasizes that Websites now represent the university and should be a key part of their knowledge management, and those who realize this will be among the beneficiaries of the new environment. It also states that citations are the highest honor for academics, and among the most objective measures of the importance of their works, a form of peer review.
Name: Google Scholar
Homepage (search): http://scholar.google.com/
Citations (and how to set up an author profile): http://scholar.google.com/intl/en/scholar/citations.html
Key characteristics: Its measure of citations is used by university ranking systems. Its search engine does not find all citations. It shows the importance of citations, links, repositories, and open Web publishing. (See more about Google Scholar in the concluding section below.)
Brief Discussion of Higher Education Rankings
The above seem to be among the most prominent higher education ranking systems, with G-Factor and Google Scholar serving as tools to measure academic output. As a reflection of their importance, they each draw critical analysis of their conclusions. There may be biases due to language, wealth, past laurels, or publicity. For instance, English has become an international language of scholarship as well as mass media. Institutions first need recognition before their merits can be duly evaluated. Thus the grounds for comparison cannot be a completely level playing field for objective analysis. High ratings attract further wealth and talent, becoming to an extent a self-fulfilling prophesy, while lesser known institutions lose talent and prestige by comparison.
As one example of controversy, European researchers not surprisingly find bias or insufficient recognition of European universities in two of the above ranking systems:
Robustness analysis of the Jiao Tong and THES ranking carried out by JRC researchers, and of an ad hoc created Jiao Tong-THES hybrid, shows that both measures fail when it comes to assessing Europe’s universities. Jiao Tong is only robust in the identification of the top performers, on either side of the Atlantic, but quite unreliable on the ordering of all other institutes. Furthermore Jiao Tong focuses only on the research performance of universities, and hence is based on the strong assumption that research is a universal proxy for education. THES is a step in the right direction in that it includes some measure of education quality, but is otherwise fragile in its ranking, undeniably biased towards British institutes and somehow inconsistent in the relation between subjective variables (from surveys) and objective data (e.g. citations). (Saisana & D’Hombres, 2008)
On the other hand, accepting the rankings as a given terrain, one could look for specific strategies, for example, along the lines of search engine optimization (Kelly, 2012).
Web Presence and Academic Recognition
As Academia is increasingly reconstituted in cyberspace (McCarty, 1997), Web presence is becoming synonymous with academic recognition, both for individuals and institutions. The increasing weight of Web presence in the higher education ranking criteria is only one reason to develop effective Web practices. The alternative is difficulty of navigation, broken links, and invisibility of university assets off campus servers or hidden in password-protected sites. The ranking systems and the whole academic ecosystem either ignore such sites or cannot find them, so the institution underperforms in the criteria that are spelled out by the organizations reviewed above. Universities are ranked above or below what close observers see, because much of the measurement is Web-based and automated. If a university is ‘punching below its weight’ in the rankings, chances are that it has been unskilled in Web development or too closed in guarding its intellectual property.
Web presence by definition involves being found and recognized, with the intellectual output of the faculty needing to be matched with the institution in the case of rankings. Actually, even a small institution that cannot generate the volume of output to be ranked should still follow good practices to not confuse but rather educate their visitors on the Web. Therefore, generally an institution or individual scholar is advised to maximize output on the open Web. Sites should be fully linked and, by the same token, maintain unchanging URLs. Universities can also utilize online repositories, which assure that publications of faculty members are fed into the databases utilized by ranking organizations or other evaluating entities. Higher education ranking systems chiefly utilize the Google search engine to measure the Web presence of institutions, so some institutions are no doubt engaging in search engine optimization (SEO), though it is no substitute for objectifying academic research and communication.
Google Scholar for Individual Web Presence
Google Scholar is introduced here as one of the most effective approaches that can be taken to enhance the academic recognition of individuals, and in the aggregate to possibly affect the evaluation of one’s institution positively. Higher education ranking systems increasingly analyze data from Google Scholar and particularly the number of citations it finds to the works of faculty members in order to measure the quantity and quality of their research output. Citations are considered a form of peer review and thus a gold standard for academic accomplishments. Thus this paper will conclude by recommending that educators establish a Google Scholar Profile. A future article will be needed to focus on Google Scholar and similar approaches, including a tutorial, because there are quite a number of steps that individuals can take besides the processes that are done automatically by Google. One must set up the profile to be able to add to and edit entries, which can be raw and inaccurate otherwise. The Help page on how to set up one’s profile is at: http://scholar.google.com/intl/en/scholar/citations.html
As mentioned, Google Scholar and particularly the aggregate citations it finds are increasingly used to rank colleges and universities. Google Scholar does not disclose its algorithms, but one can observe what it seems to do. Registered publishers, publications like journals, and university repositories funnel details on their books and articles, plus URLs where applicable, to Google Scholar, which then matches data with authors, and counts the citations it finds. It does not find all citations, but works with the online information from the publications in its database, and somehow also estimates the number of citations in some cases. It makes mistakes, too, for example when automatically matching publications to scholars with the same name. Profile users should delete such entries and try to ensure that the correct name of one’s institution is in proximity with one’s name in order to be matched by Google’s algorithm. Faculty members can add publications which are neither in repositories nor found by Google Scholar, which includes presentations and probably most publications in the world. There are a number of categories for entries that users can add, including presentations. If the chance of Google Scholar finding citations to presentations or minor publications is slight, the profile can still be used as a convenient online list of publications. It may recognize only a fraction of an individual’s citations because of its limited database or because it utilizes criteria for quality or importance, similar to how universities decide what counts as an academic publication.
Further details would be better understood with accompanying illustrations in a future article. Utilizing the Help page above, it is recommended to set up and optimize a Google Scholar profile, invite co-authors, and learn more about how it works, in conjunction with the above information about higher education ranking systems. Although it is time-consuming, there are other ways to list publications and encourage citations, such as GetCited, which feeds into Google Scholar, Academic.edu, and other open repositories.
Asian universities catching up [Editorial] (2012, October 17). Japan Times. Retrieved from http://www.japantimes.co.jp/text/ed20121017a2.html
Kelly, B. (2012, March 2). How researchers can use inbound linking strategies to enhance access to their papers. UK Web Focus. Retrieved from http://ukwebfocus.wordpress.com/2012/03/02/how-researchers-can-use-inbound-linking-strategies-to-enhance-access-to-their-papers/
McCarty, S. (1997). Academic Websites subject to attribution ethics. Humanist Discussion Group, Vol. 11, No. 433. Centre for Computing in the Humanities, King’s College London. Retrieved from http://lists.village.virginia.edu/lists_archive/Humanist/v11/0415.html
Saisana, M. & D’Hombres, B. (2008). Higher education rankings: Robustness issues and critical assessment – How much confidence can we have in higher education rankings? JRC Scientific and Technical Reports. Joint Research Centre, European Commission. Retrieved from http://publications.jrc.ec.europa.eu/repository/bitstream/111111111/12694/1/eur23487_saisana_dhombres.pdf