Part three of the DREaM sequence took place in the British Library’s conference facilities on Monday 30 January – full details are here, presentation videos are here (for community members). This was the half way mark in a series of events that I’ve written about here, exploring a range of research methods with Library and Information Science scholars in mind. After a little scene-setting from Hazel Hall (@hazelh) we moved on to the speakers.
___
The most relevant discussion for my own research was presented by Mike Thelwall on the subject of ‘webometrics’. Mike is part of the Statistical Cybermetrics Research Group at Wolverhampton University, with much of his work based on software that is free to download (see below). Thelwall described the motivation behind this kind of work as seeking to gather and make use of some of the data being added to the web all the time, material which might be used alongside other forms of research that utilises both online and offline sources. The social web in particular could sit alongside survey data: there’s a lot of information out there and it can be quick to gather. Much will depend on the topic being studied, because those who contribute online are a self-selecting community – Twitter may be a great source of information about Twitter, but it may be representative of general opinion on other subjects.
What Thelwall and colleagues have been able to show is that data from the web, including analysis of links between websites, can reflect the degree to which different countries recognise the academic output of other nations – which parts of the world are well connected and which talk only to themselves.
Tools of the trade:
- Socscibot: hosted on the Wolverhamption Uni site. ‘SocSciBot is a Web crawler for link analysis research. It is for link analysis on a single web site or collection of sites, or for text search/analysis on a collection of sites. It can also be used in teaching, to illustrate how link analysis and search engines work.’
- Webometric Analyst: also hosted by Wolverhamption. ‘Webometric Analyst, formerly known as LexiURL Searcher, automatically analyses the impact of collections of documents or web sites, and creates network diagrams of collections of web sites. It automatically submits queries to search engines and process the results.’
- Sentisoft: also at Wolverhamption. ‘SentiStrength estimates the strength of positive and negative sentiment in short texts, even for informal language. It has human-level accuracy for short social web texts in English, except political texts.’ This is about judging the strength of feeling behind a tweet, etc. Over time more rules are being added to take account of emoticons, exaggeration and new forms of language. More can be found at CyberEmotions.
- Using some of these tools it is now possible to analyse huge amounts of data: 35 million tweets, for example. We can see which events captured the most attention, which way the sentiment fell and so on.
- Comments on YouTube can also be tracked, to see who is commenting, whether they are related to other commenters on the same videos, which subjects capture the most attention and the most varied opinions. Is there a community, or are the commenters neither linked nor associated with each other?
Thelwall also drew attention to ‘altmetrics’, an attempt to devise and use alternative means by which to recognise academic output. By utilising the available data it is now possible to widen the evidence base when judging the ‘impact’ of a paper. Now blogs, government papers and other data sources offer to shine a light on the wider utilisation of academic research. Thelwall and others have taken this further, proposing an ‘Integrated Online Impact’ (IOI) indicator that combines searches from a variety of online sources – there’s plenty of potential to take this further.
Overall the message remains that these are new forms of data analysis and should be used with some caution. At the moment their greatest strength is perhaps in identifying and drawing attention to topics worthy of further analysis. Quite when and how other forms of research are then brought in is up to the researcher.
___
Peter Beresford spoke on user involvement in research, approaching the subject from a policy perspective much of the time. As such he described a dynamic relationship between action and research, with one informing the other – a pragmatic approach perhaps. An organisation that exemplifies this approach was cited in the National Institute for Health Research (NIHR), specifically their INVOLVE project.
Whether within the health field or more generally, Beresford believes there is growing pressure for more user involvement, from government, funders, users themselves and those who act on their behalf. This might include greater consultation with user groups in setting up research, closer working in partnership and ultimately research which is led and controlled by users. Where might such an influence be felt: from funding choices to methods used, peer reviewing processes and the selection of projects to pursue.
Beresford was confident that this could have an emancipatory and democratising effect, giving voice to under-represented groups and ultimately granting them new levels of control over the research agenda. Where this is linked to policy interventions the impacts could be dramatic.
Questions for researchers therefore include…
- To what extent is there a place for user-involvement?
- Who are the ‘users’ and how might they be selected, or represented?
- How much consultation is too much?
- What sort of returns can be gained from this investment and are they sufficient?
- Are the views being expressed worth taking notice of?
- What are the costs of involving service users in your research?
- What are the benefits: will users ultimately gain from the adoption of this approach?
- In a given instance is it more than just box-ticking?
- What are the implications for the REF?
When discussing policy related research, the involvement of users will necessarily add a politicised dimension to research work. Policy is about making choices, making changes and hopefully improving people’s lives – it’s not just about fact-finding. As a result there will be differences of opinion and of course contentious decisions.
So, why do it – why involve users in research? Ultimately they speak from experience: they know what they’re talking about. And yet, this goes against the positivist grain when it lacks objectivity and distance from the subject. So to not listen to the views of user groups perhaps implies that they are seen as less reliable sources of information. Such obstacles may have contributed to the length of time it has taken for some issues to gain recognition as necessary of thorough investigation, such as child abuse cases.
Beresford closed with a comment that reticence to involve users in research doesn’t ring true with real life: we regularly turn to others because of their experiences, taking their views on board, so why not when planning and carrying out research?
___
Thomas Haigh covered plenty of ground in his discussion on historical research and transferable techniques. The initial emphasis was on history as a study of change over time, as told through stories that we tell ourselves and others. Haigh singled out some examples of ‘bad’ history, such as that which overlooks connections between the present and the past. Likewise however the problem of ‘presentism’, whereby the past is judged by the standards of the present day – a comment here was that we shouldn’t be judging what actors from the past did, but what they thought they were doing.
An interesting balance was struck between academic approaches: the historian as craftsman, somewhat protective of their methods, as opposed to those in the social sciences who like to show their working. Haigh also drew attention to the variety of historical approaches which have their own communities and styles: social, institutional, cultural, etc. As such each type of history would prioritise a different type of question and focus. It seems however that each discipline goes through a phase where it discovers its history – a point is reached where it becomes important to some that the history of their subject is recorded, before it is lost. Thus begins a process of analysis that ultimately leads to histories written by trained historians, rather than the elder statesmen of a community.
___
I was also fortunate enough to talk to Alison Brettle about systematic reviews of literature. Alison is also attending all the DREaM events and in her day job works at Salford University. She has compiled this resource for her students on the uses of literature reviews, including a discussion on systematic versus traditional reviews. The reason for my interest in this is that I expect to use a systematic approach for my PhD literature review, bringing some structure and order to this element of the research in a way that rarely features in my subject area.
___
Overall another excellent day as part of the DREaM series. Next stop is Edinburgh again a little after Easter.