Seminar 21 March 2019
Title: Research Quality and Evaluation in the Humanities and Social Sciences
The KNOWSCIENCE research group will hold a seminar discussing the evaluation of HSS research. The guest speaker is Dr. Michael Ochsner from ETH Zürich and FORS Lausanne, Switzerland. He is a member of the European Network for Research Evaluation in the Social Sciences and Humanities.
Time: 10h15 - 11h30
Venue: Kulturen in Lund (Hornbergssalen)
Keynote speech: National Research Evaluation Systems, Research Quality and the SSH
Several developments during the last decades led to the emergence of systematic evaluation procedures all over Europe (and beyond). On the one hand, the transition to the knowledge society increases the importance of the universities, and research in general. On the other hand, the “new public management” paradigm calls for systematic information on the “value for money” of universities and for evidence-based steering of public institutions, such as universities. While at first, the natural, life and technical sciences were most affected by these developments, they lately affect also the social sciences and humanities.
Evaluation procedures influence not only how managers organise universities, they also affect how research is conducted and how it is perceived by the wider public. In my presentation, I will address three questions that have until now remained largely unanswered: How do evaluation procedures differ between countries? How do evaluation procedures relate to conceptions of research quality? And how do evaluation procedures allow for the characteristics of SSH research?
I will address these questions using insights from the the European Network for Research Evaluation in the Social Sciences and Humanities (ENRESSH). Our research shows that evaluation procedures are very diverse across European countries. While different procedures have advantages and disadvantages, the characteristics of a “national evaluation system” follow tradition rather than research policy needs. Often, SSH research is evaluated using STEM-oriented methods, which comes with the risk of reducing the interaction with society. A link to conceptions of research quality can help design evaluation systems that better adapt to the current situation in a country and to policy needs – not only in the SSH.
Seminar 31 August 2018
Title: Mapping institutional shaping of research priorities
Dr. Ismael Ràfols will be the guest speaker at the next KNOWSCIENCE seminar. Abstract and biosketch are below.
Time: 13h30 - 15h30
Venue: FP seminar room, 4th floor, Alfa 2
There is widespread perception that research priorities are misaligned with many of the pressing societal problems (Sarewitz and Pielke, 2017). Previous scholarship has shown that this is, at least, partly due to how a variety of social and economic factors shape research and innovation (R&I) in certain directions while left other potential trajectories ‘undone’ (Hess, 2016).
In this presentation, I will present ‘research portfolios’ as a methodological tool to map current R&I priorities for a given issue, and for particular organisations or institutions. We see this tool as means to foster active engagement and practical experiments in policy to open up space of reflection and discussions on priority setting within funding agencies.
I will offer examples from the following issues: obesity (Cassi et al. 2017), avian influenza (Wallace and Rafols, 2018), rice research (Ciarli and Rafols, 2018), and disease research prioritisation in global health (Yegros et al., 2018). For each issue, we rely on bibliometric records and innovative mapping techniques to describe existing priorities in science production using various analytical schemes. We can also illustrate the potential influence of diverse institutions in the R&I trajectories via funding, performing organisations or publishing preferences.
We propose that this type of mapping methods can be useful to domain experts for informing strategic planning and evaluation in funding agencies. We believe thatthese methods can be used in close collaboration with domain experts as part of larger methodologies that includes deliberation with diverse expertise and stakeholders.
This presentation will build on these recent contributions:
Ciarli, T. and Rafols, I. (Forthcoming) The Relation between Research Priorities and Societal Demands: The Case of Rice. Research Policy. Available at SSRN.
Rafols, I. and Stilgoe, J. (2018) Who benefits from biomedical science? Blog at The Guardian.
Wallace, M.L., Rafols, I., (online). Shaping the agenda of a grand challenge: Institutional mediation of priorities in avian influenza research. Research Policy. Available at SSRN
is a science policy analyst at (CSIC-UPV, Universitat Politènica de València), visiting professor at (University of Leiden) and associate faculty at (Science Policy Research Unit at the University of Sussex), where he worked for 8 years. He develops methods towards for informing evaluation, funding and research strategies. Ismael received a PhD in biophysics from Tohoku University (Sendai, Japan) and was postdoctoral researcher in nanobiotechnology at Cornell University. Currently, he is working on so as to facilitate deliberation on research prioritization for grand challenges such as or . He is also exploring , to correct for indicator biases and their effects, for example regarding .
More info at:
Seminar 14 June 2018
Title: Governing ‘impact’ in SSH research: implications for academic knowledge production and professional/ organisational identities
The KNOWSCIENCE research group is pleased to welcome Dr. Corina Balaban, postdoctoral research associate at the Manchester Institute of Innovation Research. Corina will present her current work on the impact of SSH research.
Time: 10h - 12h
Venue: FP seminar room, 4th floor, Alfa 2
As a sector, higher education is now a very diverse environment bringing together a whole range of actors – from academics and other university staff to governments, policy-makers, funding agencies, research councils, business and others. The boundaries between universities, government, industry and other stakeholders have become increasingly blurred, with non-academic actors getting more and more involved in setting priority research areas, determining research agendas and articulating societal ‘(grand) challenges’ to be addressed. In this context, universities are often framed as ‘problem-solving’ organisations, expected to provide quick answers to a variety of economic and societal issues. While historically there have always been expectations that universities give something back to their societies and demonstrate their public accountability, there are now increasing pressures from various policy actors for research to have a more direct, measurable societal ‘impact’. This presentation provides a critical analysis of these developments, by reconsidering questions about the role of SSH research in contemporary societies. It investigates the ways in which impact is governed (conceptualised and enacted), how certain narratives about impact influence academic knowledge production, and how these narratives ultimately shape professional and organisational identities. The study draws on organisational theory, taking a dynamic view of social change as a multi-actor, agent-centred process. It focuses on the UK and Switzerland, with the first round of data collection planned for summer 2018 to include interviews with academics and university leaders working at two selected universities in each country, and across four study areas within SSH. These will be analysed alongside key impact policies, as well as interviews with relevant (national-level) stakeholders in each country, such as research councils and funding agencies.
Seminar 23 November 2017
Title: Between the Frying Pan and the Fire: constituting notions of research excellence
The KNOWSCIENCE research group will hold a seminar discussing how notions of research quality are constituted. The seminar will feature Dr. Maria Nedeva, Guest Professor in the Strategy and Research Policy Group at Lund University. She is also Professor of Science and Innovation Dynamics and Policy at Alliance Manchester University.
Time: 13h30 - 15h30
Venue: EC1:136, Holger Crafoords Ekonomicentrum, Tycho Brahes väg 1, Lund
During this seminar, we’d revisit the highly contextual, complex and dynamic nature of the notion(s) of research quality and discuss a framework for its study and understanding. This framework uses two sets of assumptions. One of these builds on existing studies of the notions of research quality to explicate three characteristics that inform notions of research quality (Polanyi, 1962; Hemlin, 1991; Lamont, 2009; Bazeley, 2010; Mårtensson, 2016). Another set of assumptions uses contemporary understanding of the science system (Whitley, 2000; Whitley et al. 2010; Glaser et al. 2010, Nedeva, 2013), and its dynamics, to identify five organisational contexts where notions of research quality are constituted.
Imperatives for increased levels of accountability, legitimacy and growth have contributed to widening the concept of research quality in policy and funding circles. Furthermore, the power to define research quality has now shifted from being the sole province of knowledge communities (research fields) to being a shared responsibility of the research community and significant others.
This shift of epistemic authority from science has implications for the way in which we understand and study research quality. This paper outlines a new approach to the study of research quality which takes its point of departure in the processes and mechanisms through which dominant notions of research quality are developed and established.
(This research is being carried out by an international team that includes colleagues from Norway, the Netherlands, Sweden and the UK and supported by the Research Council of Norway.)
Conference 7- 9 June 2017
KNOWSCIENCE is co-hosting a session at the Eu-SPRI meeting in Vienna under the title: The impact of STI-policies on the organization and production of knowledge. See link for details.
Seminar 24 May 2017
Dr. Aant Elzinga will be the guest speaker in the next KNOWSCIENCE seminar. Aant is from the Department of Philosophy, Languistics, and Theory of Science at the University of Gothenburg. He will be presenting his work: The Cold War, Political Neutrality, and Academic Boundaries: Imprints on the Origins and Early Development of Science Studies in Sweden.
Time: 13h00 - 15h00
Venue: Rhenmansalen, Alfa2, Scheelevägen 15A, Lund
Seminar 19 April 2017
The KNOWSCIENCE project group is pleased to welcome Drs. Ruth Mueller and Wolfgang Kaltenbrunner from the Munich Center for Technology in Society, Technical University Munich. Ruth and Wolfgang will share some of the latest findings from their respective research projects with us in a mini workshop on the 19/4 at 13h.
Rhenmansalen, Alfa2, Scheelevägen 15A, Lund
13.15 Dr. Ruth Mueller: A Lab of One’s Own? On Becoming Independent in Contemporary Academia
14.15 Coffee Break
14.20 Dr. Wolfgang Kaltenbrunner: Situated knowledge, international impact: Changing publishing practices in a German automotive engineering department
15.30 Wrap up
Workshop 24-25 November 2016
Title: Globalising practices in the governance of higher education and research
Globalising Practices in the Governance of Higher Education and Research is a research programme that bridges the divide between research on higher education and research policy. The programme is built around three cases which together shed light on how the practices that now dominate the governance of science disseminate in the European context and beyond. The cases explored are the role of the ESF in the diffusion of governance practices; the development of global rankings of higher education and the evolution of competitive funding for basic research in the European Research Area.
Keynote speaker: Benedetto Lepori
Professor Benedetto Lepori holds a PhD in Communication Sciences with a thesis on the Swiss research and higher education policy at the University of Lugano in 2004. At the University of Lugano,he is titular professor at the Faculty of Communication science, Institute of Interdisciplinary Data Sciences. He is also head of the Research Service of the University and of the Scuola Professionale della Svizzera italiana.
His research interests cover a broad range of topics in research and higher education studies. He is a recognized specialist in the analysis of research policies and, especially, public research funding. In this area, he coordinated the PRIME project on public project funding and was co-coordinator of the European Contract on Joint and Open Programs (JOREP). He has also worked extensively in the domain of higher education indicators and governance; he participated in PRIME-AQUAMETH and the European MIcroData study, and is currently coordinator of the European Tertiary Education Register (ETER). He was also chair of the European Network of Indicator Designers conference series and is currently member of the Coordination Board of the RISIS project on developing research infrastructure for research policy studies. More generally, his research focuses on issues of higher education governance, university management and theory of S&T indicators.
He has published extensively on these topics in journals like Organization Studies, Research Policy, Science and Public Policy, Research Evaluation, Evaluation, Journal of Informetrics, Scientometrics, Higher Education and Studies in Higher Education.
Keynote speech: US and European universities. Is the difference money?
Large differences in the level of scientific production and of international reputation between top-US and European universities are well-known, as represented for example by international research rankings (Albarrán, Crespo, Ortuno and Ruiz-Castillo 2010). It has been speculated that these differences are generated by different governance and funding in the two systems, with the US system promoting more competition and differentiation and concentrating funding in top-ranked universities (Bonaccorsi 2009; Aghion, Dewatripont, Hoxby, Mas-Colell and Sapir 2010). This paper proposes an empirical analysis of such differences. By matching data from the US Integrated Postsecondary Education Data System (IPEDS; http://nces.ed.gov/ipeds/) and the European Tertiary Education Register dataset (ETER; www.eter-project.com), enriched with bibliometric data from the Leiden Ranking, we are able to provide a disaggregated analysis of differences in characteristics and level of funding of individual Higher Education Institutions (HEIs) in both systems. We move beyond comparisons based on aggregated figures and averages to focus on differences in the distributions across the two systems and relevant variables.
Our analysis displays a complex pattern of differences across different dimensions. The US system is more diverse in terms of institutional status and types of HEIs, while in Europe universities constitute the core of the system both in research and in education. In terms of activities, the US system is more distributed in terms of education than Europe, but much more concentrated for what concerns research. We highlight two strong differences in the funding system: first, the differentiation of funding channels in the US (State contributions, federal research grants, donations, tuition fees) as contrasted with the dominance of the government core allocation in Europe and, second, the much stronger concentration of resources in the top-US universities, whose budget is a multiple of the best-funded European universities. We also show that the volume of research output (measured through scientific publications) increases almost linearly with the funding volume, while reputation (measured through citations) scales more than proportionally with funding (van Raan 2013), implying that stronger concentration of funding is associated with higher international reputation. Implications of these findings for public policies are finally discussed.
Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A. & Sapir, A. (2010). The governance and performance of universities: evidence from Europe and the US. Economic Policy, 25(61), 7-59.
Albarrán, P., Crespo, J. A., Ortuno, I. & Ruiz-Castillo, J. (2010). A comparison of the scientific performance of the US and the European Union at the turn of the 21st century. Scientometrics, 85(1), 329-344.
Bonaccorsi, A. (2009). Division of academic labour is limited by the size of the market. strategy and differentiation of european universities in doctoral education. In M. McKelvey & M. Holmén(Eds.) Learning to compete in European universities (pp. 90-127). Cheltenham: Edward Elgar.
van Raan, A. F. (2013). Universities scale like cities. PloS One, 8(3), e59384.