Wednesday, 2 April 2014

PollEverywhere Higher Education License Pricing

PollEverywhere is a 'bring your own device' audience response system. Your audience can respond by SMS or web (or Twitter!). Although we have access to clickers we found that they were not used as much as they could be - in part because of the extra effort of getting the clickers, handing them out and then getting them back. Last year one of my colleagues suggested that we look into PollEverywhere as it had been used by the university for public engagement events.

A free PollEverywhere license allows up to 40 responses to any question or poll. But we have over 300+ students in the 5 years of our undergraduate course so I approached PollEverywhere last year to find out how much a HigherEd license would be for 1500 students. I was advised that pricing was based on maximum number of responses not on students. This made sense since we would not actually be registering or tracking individual students on the system. So we purchased a license which allowed us to have up to 330 responses to each poll. This was priced at $3.50 per student response. The pricing model made sense because it felt as if we were buying 330 virtual clickers. This license allowed us to have an unlimited number of tutor accounts. We started with 50 but haven't needed to go beyond that yet.

We've been learning a lot as we have gone through the year about the pros and cons of using the various question types and I and others have used this license at various public events with great success. Last week we were contacted by PollEverywhere to inform us that we had gone outside the terms of our license. I have spoken to the PollEverywhere team today and they estimate that we have had 829 unique responders/students  to our polls in the last 2 months. They do this by applying an algorithim looking at IP addresses and unique mobile numbers. This isn't in any way surprising as we had a public engagement event a few weeks ago and have also been demonstrating PollEverywhere at teaching events and conferences.

PollEverywhere say that there has been a misunderstanding and that we have been purchasing unique accounts for our students not a maximum number of responses to a given poll. I asked about public engagement activities- would be need to purchase a separate license for these? They say that we don't need to if we email them in advance and let them know that we will be using the license to a public audience.

By the end of the week PollEverywhere are going to clarify the terms of the Higher Education license to us in writing. This documentation is being written now. They don't plan to put this on their website as they say that there has not been a need as there have been no other misunderstandings about this. Since I told many people about what I thought was the terms of our existing license I thought I would write this blog post to explain that my understanding of the license has now changed.

We now have to decide whether we want to purchase this license again considering that it will be 3-4times as expensive next year as this year. We will review other options such as Participoll. It may be back to clickers after all!

Thursday, 20 March 2014

very quick survey about Facebook in education

social media in medical education... it's all about the network

What is the role of Twitter in medical education?

I started using Twitter (and this blog) because I wanted to connect with people who I didn't know how to connect with otherwise. I wanted to connect to people who worked in medical education but I got a lot more than I bargained for. I found a wonderful network of people who I continually learn from. I can share, and get feedback, dip in and out, refine my thoughts, and the rest.

But do I think that we should use it in our courses? What for?

Last week I attended a workshop lead by Claudia Megele on social media in higher education. She has done tremendous work in establishing several 'knowledge networks' including #mhchat (mental health chat). She then introduced a group of social work students to social media including Twitter. They used pseudonyms but participated in regular Twitter chats and discussed topics online.

It seems to me that the most powerful thing that Claudia did was to introduce her students to her networks. She helped them to become part of a community discussing mutual topics of interest online.

So how can we, committed to health professional education, help our students? I think we can do the same thing. We can give our students access to our knowledge networks.

Wednesday, 12 March 2014

The challenge of feedback and the burden of accountability in clinical medical education

Have you received good feedback in a clinical setting as a doctor or medical student?

Some of the best feedback I received was during my GP reg training year when my trainer reviewed most of my consultations with me every day for the first 4 or 5 months. I knew it would be a chance to ask what he thought about a patient, and also that I would have to justify some of the decisions I had made or hadn't made. It was above and beyond any feedback I have ever had before or since on my clinical work.

We know what makes good feedback. Here is a great paper about this from 31 years ago. There was no talk about eportfolios or skills logs or apps at that time. 16 years later when I was during my GP trainee year we didn't have the burden of documenting our feedback either.

So do current moves to use technology, including smart phones, to try and document feedback in clinical settings enhance the quality of feedback given? Do they make it more likely to happen? If they don't why do we do them?

Is there a risk that in an effort to be accountable, we are making it harder for learners to achieve good feedback because of the burden of documentation? If you want to read more about this I strongly recommend Onora O'Neill on 'Intelligent Accountability in Education'.

Thursday, 27 February 2014

Politicians badly briefed about data issues too?

At the end of the Parliamentary meeting on 'Patient rights and access to NHS data',
Parliamentary Under Secretary of State for Public Health, Jane Ellison MP made the following statement:

"I should actually just before we close put on the record Mr Emerson, forgive me, but I think it is useful for colleagues, just in regard to the Faculty of Actuaries and the data there, and I think actually the Shadow Minister also alluded to this; just to put on the public record that the data that they used was publicly available, non-identifiable and in aggregate form."

This data as described in this blog post was not publicly available or in aggregate form. It was individual-level data that had to be specifically asked for from the NHSIC. 

Jane Ellison may have been referring to the report generated from the research done which is indeed publicly available, with aggregate analyses from which it would be impossible to identify an individual. But no one has concerns about the report.

You can watch her statement here at 16.16.30

Wednesday, 26 February 2014

We need data literate journalists.

So what do you think happened recently with NHS data? Do you think that the NHS handed over the records of millions of patients to insurers who then looked up their credit records and suggested that their insurance premiums should be changed?

That would be awful wouldn't it? Just as well it didn't happen.

I came back from visiting family on Sunday night to see the  Telegraph  story "Hospital records of all NHS patients sold to insurers" being tweeted.  It was picked up by many other papers including the Guardian.

The story is about research done by a group of actuaries, the Critical Illness Definitions and Geographical Variations Working Party . You can find the full report here.  It's a 200 page plus document written to explain to other actuaries how 'geodemographic' data might help predict how likely someone is to develop a critical illness.  If you have ever applied for life insurance or critical illness cover or an income protection policy you would know that you your policy is priced based on what your risk is... your age, weight, smoking status, what illnesses you have. If you know anything about health inequalities you will know that beyond our own personal risk factors( age, weight etc) our social circumstances are important in determining how long we will live and if we will get sick. And you can tell a lot about our social circumstances from where we live. The relationship is so strong that there is a postcode mortality lottery Your postcode *might* reflect your lifestyle, your wealth, your education- all the things that predict how likely you are to get sick or to live long.

Geodemographics , CACI's Acorn, and Experian's Mosaic , classify postcodes into strange-sounding groups like 'happy families' and 'twilight subsistence' based on information obtained from public and commercial sources. The research by the actuaries was about whether these postcode classifications could predict when people developed serious illnesses. You can read the report to find out more but the short answer is that they do.

You may disagree with the idea that you postcode should be used to predict your risk to insurers. Is it a smart way of doing things? You can read some discussion of this in Tony Hirst's blog post here.

Most of the discussion was not about this though. It was about the fact that 'hospital records' were given to insurers.

So what actually happened in the research? 

The above tweet by Roger quotes the Guardian's coverage of this story. Are Acorn and Mosaic 'credit ratings data'? Well, yes, they may have been originally  developed to predict how likely you were to be able to pay back a loan. But as we can see they can also predict how likely you are to get sick or to die.

What did the hospital records look like? There were Hospital Episode Statistics. This is what the data looked like (from this presentation)
Is that what you thought the 'hospital records' would look like?

Did the researchers have full postcodes and dates of birth? It was a bit hard to tell this from the report. I presumed they didn't because I didn't see why they needed it. And I didn't think that the NHS was likely to give away information that would make it easy for individuals to be re-identified. But the full postcode was needed to be able to assign a 'geodemographic profile' to each person in the dataset. The following screenshot is from page 10 of the report.

I read this as meaning that the geodemographics were added to the HES dataset by the NHSIC who provided the dataset to the insurers. But others, including Tony Hirst, first read this as meaning that it was the researchers that did the datalinking. Who did the datalinking was important because who ever did it needed the full postcode. 

Today after reading an article by Wired in which it is stated the hospital data was given to the Institute and Faculty of Actuarie IFOA and "was then combined with secondary sources, including Experian credit ratings data, in order to influence insurance premiums." I decided that I had to find out. So I phoned the press office of the Institute and Faculty of Actuaries (IFOA) on the number I found on the press release of their rebuttal to the Telegraph article.

I got straight through. The press officer directed me to page 10 above. I asked who had done the datalinking and they said it was the NHSIC. This made sense and fitted with their statement that they had no identifiable information for the individuals in the dataset. They only had an age group, and the 1st part of their postcode.

So how many people do you think contacted the IFOA to try and make the same clarifications as me? Every journalist that had written a story about this perhaps? No 3 people. The BBC and two bloggers. I was one of them. 

Why didn't other journalists get in touch with them? Didn't they understand the significance of this? Didn't they care?

In the next few months and years we are going to be having many conversations about big data. We need to have journalists who know how to ask the right questions. And at the moment it looks as if we haven't.

If you think that the problem is that actuaries were given NHS data at all then see this.
EDIT In the past GPRD data was provided to actuaries. This is no longer the case although at least one application was made recently to CPRD. They rejected this.