The systematic quantitative literature review is a smart and effective method for undertaking literature reviews particularly for research students and others new to a discipline.
Reliable, quantifiable and reproducible
It bridges the gap between traditional narrative review methods and meta-analysis. Narrative methods that are commonly used in many research theses, rely on the expertise and experience of the author, making them challenging for novices. In contrast, the method we use and recommend involves systematically searching the literature using online database and other sources to find all relevant papers that fit specific criteria (systematically identifying the literature), entering information about each study into a personal database, then compiling tables that summarise the current status of the literature (quantifying the literature). The results are reliable, quantifiable and reproducible. Using this method, it’s also possible to determine if there are suitable datasets for meta-analysis. By mapping the literature we can not only identify what is known, but also, but where there are gaps: a critical issue in advancing research and designing PhD research programs.
The method works well for specific topics, but also for summarising diverse inter-disciplinary research. Using this method many of our students and others have gone on to publish their reviews. Importantly for PhD students, the database can be updated during the PhD thesis allowing them to easily identify relevant papers and produce their final thesis without having to re-read all the literature.
Publishing research papers in science
- Pickering, C.M. (2012). Writing Ecology Research Papers. Environment Futures Research Centre. Griffith University, Gold Coast
Our papers outlining the method and its benefits
- Pickering, C.M. and Byrne, J. (2014). The benefits of publishing systematic quantitative literature reviews for PhD candidates and other early career researchers. Higher Education Research and Development. 33: 534-548
- Pickering, C., Grignon, J., Steven, R., Guitart, D. and Byrne. J. (2015). Publishing not perishing: How research students transition from novice to knowledgeable using systematic quantitative literature reviews. Studies in Higher Education. 40:10, 1756-1769, DOI: 10.1080/03075079.2014.914907. A pre-print copy of the paper is available here, or the final published version from the publishers website for the online version
Article in The Conversation on the method:
- Pickering, C.M. and Byrne, J. (2014). How to find the knowns and unknowns in any research. The Conversation.
Important reference for how to report on how you systematically searched for papers that may be required by journals:
- Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097
Some other useful references on literature reviews
- Boote, B.N. and Beile P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher. 34: 3-15.
- Crisp, B.R. (2015) Systematic reviews: A social work perspective. Australian Social Work, 68:3, 284-295.
- Murray, R. (2011). How to Write a Thesis. McGraw Hill Open University Press. Maidenhead, England (Chapter on writing a literature reviews).
- Petticrew, M. and Roberts, H. (2006). Systematic Reviews in the Social Sciences: A Practical Guide. Blackwell Publishing, Oxford, England.
- Randolph J.J. (2009). A guide to writing the dissertation literature review. Practical Assessment, Research and Evaluation. 14: 1-13.
Example excel databases of Systematic Quantitative Literature Reviews
Here are examples of the types of excel databases used in some Systematic Quantitative Literature Reviews:
- Steven et al. 2011 database of papers on impacts nature based tourism on birds (XLSX 16KB)
- Ballantyne and Pickering In review databases of papers on environmental impacts of recreation trails (XLSX 32KB)
- Byrne and Portanger 2014 database of papers climate change, energy policy and justice (XLSX 48KB)
Examples of papers published using this method
- Ansong, M. and Pickering, C.M. (2013). A global review of weeds that can germinate from horse dung. Ecological Management and Restoration. 14: 216-223
- Ansong, M. and Pickering. C.M. (2013). Are weeds hitchhiking a ride on your car? A systematic review of seed dispersal on cars. PlosOne. 8:e80275
- Ansong, M. and Pickering. C.M. (2014). Weed seed on clothing: A global review. Journal of Environmental Management. 144: 203-211
- Ballantyne, M. and Pickering, C.M. (2013). Tourism and recreation: A common threat to IUCN red-listed vascular plants in Europe. Biodiversity and Conservation. 22: 3027-3044. DOI 10.1007/s10531-013-0569-2
- Ballantyne, M. and Pickering C.M. (2015). The impacts of trail infrastructure on vegetation and soils: current literature and future directions. Journal of Environmental Management. 164: 53-64. 10.1016/j.jenvman.2015.08.032
- Bartesaghi Koc, C., Osmond, P. and Peters, A. (2016). Towards a comprehensive green infrastructure typology: a systematic review of approaches, methods and typologies. Urban Ecosystems. 1-21. doi:10.1007/s11252-016-0578-5
- Barros, A. Monz, C. and Pickering, C.M. (2015). Is tourism damaging ecosystems in the Andes? Current knowledge and an agenda for future research. Ambio. 44:82–98. DOI 10.1007/s13280-014-0550-7
- Bergstrom, J., van Winsen, R. and Henriqson, E. (2015). Reliability engineering and system safety. Reliability of Engineering and System Safety. 141:131-141
- Botzat, A., Fisher, L.K. and Kowarik, I. (2016). Unexploited opportunities in understanding liveability and biodiverse cities. A review of urban biodiversity perception and valuation. Global Environmental Change. 39: 220-233
- Boulton, C., Byrne, J., Dedekorkut-Howes, A., (2015). The role of political and financial factors in the provision of parks: The case of Logan City, Queensland. Sate of Australian Cities Conference
- Byrne, J. and Portanger, C. (2014). Climate change, energy policy and justice: A systematic Review. Analyse & Kritik. 36:315-418
- Chester, P., Ehrlich, C, Warburton, L, Baker, D., Kendall, E. and Crompton, D. (2016). What is the work of recovery orientated practice? A systematic literature review. International Journal of Mental Health Nursing. 25: 270-285
- Cheng, M., Edwards, D., Darcy, S., and Redfern, K. (2016). A tri-method approach to a review of adventure tourism literature: Bibliometric analysis, content analysis and a quantitative systematic literature review. Journal of Hospitality and Tourism Research. In Press
- Clark, N.J., Clegg, S.M. and Lima, M.R. (2014). A review of global diversity in avian haemosporidians (Plasmodium and Haemoproteus: Haemosporida): new insights from molecular data. International Journal of Parasitology. 44: 329-338
- Crisp B. (2015) Systematic reviews: A social work perspective, Australian Social Work. 68:3, 284-295, DOI: 10.1080/0312407X.2015.1024266
- Davis N.E., Bennett A., Forsyth, D.M., Bowman D.M.J.S., Lefroy E.D., Wood S.W., Woolnough A.P., West, P., Jordan O. Hampton, J.O. and Johnson C.N. (2016). A systematic review of the impacts and management of introduced deer (family Cervidae) in Australia. Wildlife Research, 43(6): 515-532 http://dx.doi.org/10.1071/WR16148
- De Brito, M.M and Evers, M. (2015). Review article: Multi-criteria decision making for flood risk management: a survey of the current state-of-the-art. Natural Hazards and Earth Sciences. 3, 6689-6726
- Emam, H., Farrell, P. and Abdelaal, M.A. (2014). Causes of delay in GCC construction projects: a Critical Review. Paper published in the 1st Int’l Conference on Smart, Sustainable and Healthy Cities. Abu Dhabi, UAE. pp 607-621
- Falkenberg, L.J. and Tubb, A. (2017). Economic effects of ocean acidification: Publication patterns and directions for future research. Ambio 1-11. doi:10.1007/s13280-017-0895-9
- Finlayson, K.A., Leusch, F.D.L. and van der Merwe, J.P. (2016). The current state and future directions of marine turtle toxicology research. Environmental International. 94: 113-123
- Gstaettner, A.M. Lee, D. and Rodger, K. (2016). The concept of risk in nature-based tourism and recreation – a systematic literature review. Current Issues in Tourism. pp 1-26
- Guitart, D., Pickering, C.M., Byrne, J. (2012). Past results and future directions in urban community gardens research. Urban Forestry and Urban Greening. 11: 364-373
- Hains-Wesson, R., Tytler, R. (2015). A perspective on supporting STEM academics with blended learning at an Australian university. Issues In Educational Research. 25: 460-479
- Hegetschweilera, K.T., de Vriesb, S., Arnbergerc, A., Belld, S., Brennane, M., Siterf, N., Olafssong, A.S., Voigth, A. and Hunzikera, M. (2017). Linking demand and supply factors in identifying cultural ecosystem services of urban green infrastructures: A review of European studies. Urban Forestry & Urban Greening. 21: 48-59.
- Herington, M.J., van de Fliertb, E., Smarta, S., Greiga, C. and Lanta P.A. (2017). Rural energy planning remains out-of-step with contemporary paradigms of energy access and development. Renewable and Sustainable Energy Reviews 67: 1412–1419.
- Hirsch, P.E., N’Guyen, A., Adrian-Kalchhauser, I. and Burkhardt-holm, P. (2015). What do we really know about the impacts of one of the 100 worst invaders in Europe? A reality check. Ambio. DOI 10.1007/s13280-015-0718-9
- Howes, M., Wortley, L., Potts, R., Dedekorkut-Howes, A., Serrao-Neumann, S., Davidsone, J. Smith, T. and Nunn. T. (2017). Environmental Sustainability: A case of policy implementation failure? Sustainability. 9(2): 165; doi: 10.3390/su9020165.
- Jasinski, D., Meredith, J., and Kirwan, K. (2015). A comprehensive review of full cost accounting methods and their applicability to the automotive industry. Journal of Cleaner Production. 108: 1123–1139
- Kollmann, J., Myer, S.T., Bateman, R., et al. (2016). Integrating ecosystem functions into restoration ecology – recent advances and future directions. Restoration Ecology. In Press 10.1111/rec.12422
- Leung, J., Ferrari, A., Baxter, A., Schoultz, M., Beattie, M. and Harris M. (2017). Systematic reviews: inducting research students into scholarly conversations? Higher Education Research & Development 36: 217-220.
- N’Guyen, A, Hirsch, P. E., Adrian-Kalchhauser, I, and Burkhardt-Holm, P. (2015). Improving invasive species management by integrating priorities and contributions of scientists and decision makers. Ambio, DOI 10.1007/s13280-015-0723-z
- McAnulty, T., Jeswiet, J. and Doolan, M. (2016). Formability in single point incremental forming: A comparative analysis of the state of the art. CIRP Journal of Manufacturing Science and Technology. 16:43-54.
- Papadonikolaki, E. Koutamanis, A. and Wamelink, J.W.F. H. (2013). Attaining performance and building information modelling: A systematic literature review of product and process modelling in AEC. Building Information Modelling. Volume 2. Computation and performance eCAADe31 475-483
- Phan, T.D., Smart, J.C.R., Capon, S.J. and Hadwen, W.L. (2016). Applications of Bayesian belief networks in water resource management: A Systematic Review. Environmental Modelling and Software. 85: 98-111
- Riebe, L., Girardi, A., and Whitsed, C. (2016). Teaching teamwork skills in Australian higher education business disciplines. Issues in Educational Research, 27
- Roche, S. (2017). Child protection and maltreatment in the Philippines: A systematic review of the literature. Asian and The Pacific Policy Studies. 10.1002/app5.167
- Roy, S., Pickering, C. and Byrne, J. (2012). A systematic quantitative review of urban tree benefits, costs, and assessment methods across cities in different climatic zones. Urban Forestry and Urban Greening. 11: 351– 363
- Rupprecht, C.D.D., Byrne, J.A, Garden, J.G. and Hero, J. (2015). Informal urban green space: A trilingual systematic review of its role for biodiversity and trends in the literature. Urban Forestry and Urban Greening. 14: 883-908
- Sato, C.F, Wood, J.T. and Lindenmayer, D.B. (2013) The effects of winter recreation on alpine and subalpine fauna: A systematic review and meta-analysis. PlosOne. 8:e64282.
- Schoenberga,, G., Cuskellya, G. and Auldb, C. (2016 In Press). The role of intragroup dynamics in non-profit governance models: a systematic quantitative literature review. Managing Sport and Leisure. DOI:10.1080/23750472.2016.1169212
- Spasojevic, B., Lohmann, G. and Scott, N. (2016). Systematic literature review on air transport and tourism: Analysis from 54 journals during the period 2000-2014. CAUTHE 2016: The Changing Landscape of Tourism and Hospitality: The Impact of Emerging Markets and Emerging Destinations. Sydney: Blue Mountains International Hotel Management School, 2016: 293-313
- Steven, R. Pickering, C.M. and Castley, G. (2011). A review of the impacts of nature based recreation on birds. Journal of Environmental Management. 92: 2287-2294
- Steven, R., Morrison, C. and Castley, J.G. (2014). Birdwatching and avitourism: A global review of research into its participant markets, distribution and impacts, highlighting future research priorities to inform sustainable avitourism management. Journal of Sustainable Tourism: online DOI:10.1080/09669582.2014.924955
- Trembath, D. Westerveld, M. and Shellshear, L. (2016). Assessing spoken language outcomes in children with ASD: a Systeamatic Review. Current Development Disorders Reports, 3: 33-45
- Thomas, S. (2014). Blue carbon: Knowledge gaps, critical issues and novel approaches. Ecological Economics, 107: 22-38
- Ward, A., Dargusch, P., Thomas, S., Liu, Y., Fulton, E.A. (2014). A global estimate of carbon stored in the world’s mountain grasslands and shrublands and implications for climate policy. Global Environmental Change, 28: 14-24
- Ward, A., Dargusch, P., Grussu, G. and Romeo, R. (2016).Using carbon finance to support climate policy objectives in high mountain ecosystems. Climate Policy, 16: 732-751. DOI:10.1080/14693062.2015.1046413
- Wortley, L. Hero, J., Howes, M. (2013). Evaluating ecological restoration success: A review of the literature. Restoration Ecology, 21: 537-543
- Yang, E.C.L., Catheryn Khoo-Lattimore, C. and Arcodia, C. (2017). A systematic literature review of risk and gender research in tourism. Tourism Management 58: 89-100
Please let us know if you do publish using this method and we can add your paper to this list.