PhD Pivot time

The big outcome from my interviews has been the realization that the subjects of my study are wedded to Facebook and that there's not much I can do about that. Each interview participant has come back saying they are using Facebook because everyone else is using it, and aren't using my tools because no-one else is. Several of my interview participants were selected specifically because of their posting frequency on my system - each posted a number of times, then stopped. The interviews revealed that they stopped because they discovered that everyone else was in Facebook, so if they wanted their posts to be seen and responded to, they needed to go there. And if they wanted to see what everyone else was sharing, then Facebook was the place to be. Perhaps in a different time, with a different cohort of students, this wouldn't be the case; but where I'm conducting my study, it's inescapable.

This result changes what I can do with regard to my study. The evidence is saying that regardless of what I do, there is no incremental improvement I can make to my system that will tip the balance - the network effects of students who are familiar with Facebook drawing the rest of the cohort into Facebook are overwhelming. This is an interesting developing in itself - it says a lot about the importance of various effects. It also says a lot about my original research questions: that to be an effective social network for learning, the space needs to be (a) private, (b) dedicated to learning, and (c) linked to curriculum.

The way the students are using Facebook indicates that (a) and (b) still apply, to some extent. Students are using a combination of "closed" (anyone can see the group, but needs approval to join) and "secret" (you need to be specifically invited to join) Facebook groups, so they are protecting their privacy in that way. In one sense, these groups are more private than the ones I can offer, since mine will always be vulnerable to Faculty oversight, whereas Facebook is unlikely to care about the contents of these groups except to algorithmically deliver advertising to the participants. These groups are dedicated to curriculum, and so my thoughts about students being concerned about friends and family not needing to be exposed to this content are still somewhat valid. However, the linking to curriculum isn't there at all. Several of my interviewees said that linking to curriculum would be valuable, but in the end that being in the same place as the other students was more important.

Which leads me to the conclusion that it may make sense to accept that I'm beaten, and start working with the enemy. I've started exploring the Facebook APIs, to see what is available. The features I'd like to be able to offer students include:

• Linking groups - students should be able to connect a group in the Medical Program to a Facebook group, so that links from one to the other can be created. The Medical Program's Portal can then give students links to the Facebook sites relevant to their groups.
• Curriculum linking - it would be good to give students the ability to easily link content shared in Facebook with curriculum within the Medical Program. How to do this is unclear - maybe a sharing tool within the Medical Program that pushed content to the right Facebook group? Each of the tools Facebook provides seems to be in some way limited that makes this difficult.

This shouldn't be the end if it, though. As my supervisor pointed out, it is a bit counterproductive to just assume that Facebook will be sufficient. What happens if the new cohort who arrive next year are wedded to another network, rather than Facebook? All it takes us a core of sufficiently active students to set up a group and make it interesting to others, to draw the rest of their cohort in one by one. It could be Slack or Pinterest or Instagram or Twitter or Evernote or Google+ (haha) or some other tool that becomes their central space for online interaction. To make this relevant and future-proof, and to prevent a system that tries to lock students into Facebook. I need to allow for extensibility, to encourage students to explore and find the right environment for them. How do do this in a rich and interactive way, while still coping with the tremendous variety of platforms out there, is an interesting question.

There are wider philosophical issues that come into play here, about what institutions should be doing and providing for their students, which I will cover in another post.

Interview progress

I'm well into my interviews - I have now interviewed five students about their use of my system and other social networks. I'm getting an interesting range of perspectives from students about how they use these tools and what they'd like to see from them.

Firstly, it's now very clear to me that the student body I'm studying is heavily invested in Facebook. All participants use Facebook for their studies, and no other social network. A couple had a presence on Academia.edu and ResearchGate, but there was no Twitter, Tumblr, Instagram, or other social networks in use, particularly not for their studies. Facebook has a clear first-mover advantage over any system I develop: the students all have Facebook accounts before they come to this degree, so it's natural to them to set up Facebook study groups. They are setting up whole-of-year and tutorial group Facebook groups, but also study groups of varying sizes and success rates. One very active study group has 260 members - nearly the whole cohort. From the way these groups are entrenched in the students' study lives, it would be very difficult to switch students over to a new system - they would need to be steered to use the new system from the beginning of first year, probably by academics warning them of the dangers of Facebook. But their patterns of Facebook usage have also indicated to me some features that are missing from the system as it stands - particularly, the ability to post to groups, rather than to the whole cohort. Interestingly, that facility was in my original design plan, but I abandoned it after deciding that I needed to maximize the exposure of each item to students in order to increase cross-cohort collaboration.

The students also had some interesting ideas about changes that would improve the system. A common thread was the request for more flexibility in their self-presentation on the profile pages. Currently, the pages allow them to present a small amount of information about their interests and previous degrees, but suggestions for additions (particularly from one participant) included travel plans, favourite books, medical specialty and placement interests. They would like to be able to better present themselves as professionals. Another common thread was that it would be useful to have less rigidity in how the resources were attached to the curriculum: rather than exclusively binding the resources to learning objectives, it would be useful to bind to other levels of the curriculum; and to search and be able to tag the resources.

The students have generally agreed that an embedded system made sense, and would be preferable to Facebook if done right, but generally pointed out that it would be difficult to do. It wasn't clear to me whether they were sincere or were just trying not to hurt my feelings - as they all knew I was the architect of the system and had a vested interest in it succeeding. But they did understand the reasoning behind linking to the curriculum, as well as having a protected space. The issue of copyright was one that several noted - they mentioned that one frequently shared item on Facebook is textbooks, and that they wouldn't be comfortable sharing those in a University controlled space.

Most students had a reasonable amount of awareness of the available functionality. But when discussing it, a few things stood out. Many didn't realize that the rating tools existed - that they could collaboratively curate the resources uploaded by their peers. And none of them were aware that staff can't see the shared resources. I had announced this early on, and I'm fairly sure that I repeated my announcement, but none of them were aware. This wasn't one of my scripted interview questions - I found out when students stated that one reason for their wariness of these tools was that staff could see what they were doing.

Lastly, I realized very quickly that after these interviews I would need to prepare some feedback to the Faculty about the students' experiences in the degree. One student stated that one of the reasons she was so keen to participate in the interviews was that it would give her a way to feedback to the University (side note here: I am also a staff member, though not in the Faculty I am studying). It seems to me that it would be an ethical breach not to give this feedback to the Faculty, so in the later interviews I have informed students that I would be collating the feedback that is relevant to the teaching of the program, and giving that to the Faculty. Quite a bit of it is relevant to my project - particularly the students' descriptions of the way assessment drives their learning behaviour, and how it limits their interest in sharing, but also in reading outside the scope of what will be assessed. It's clear that a culture has been developed within the student body, driven inadvertently by the Faculty, of learning only what will be on the exam, rather than developing a deep and broad understanding of Medicine and how it fits into our world.

These have been very informative interviews; I'm planning to interview two or three more students, but I already have a collection of things I can act on, and with luck be able to deliver to students before next year's intake starts their studies.

Groups, Networks, and Collectives

After reading

Dron, J., & Anderson, T. (2009). On the Design of Collective Applications (Vol. 4, pp. 368–374). Presented at the 2009 International Conference on Computational Science and Engineering

and

Dron, J., & Anderson, T. (2009). How the Crowd Can Teach. In S. Hatzipanagos & S. Warburton, Handbook of Research on Social Software and Developing Community Ontologies (1st ed., pp. 1–17). Hershey, PA: Information Science Publishing.

and conducting interviews, I’ve been thinking that perhaps I’m targeting the wrong layer of social entity for my social tools. The papers above discuss an important classification of ways of looking at the behaviour of sets of people.  Dron and Anderson define the group as a small social unit where people know each other (eg. a class); a network as a set of individuals linked by connections of some kind (eg. friendships, or comments on each others’ posts), and a collective as something that emerges from the mass actions of a large group of people. They aren’t mutually exclusive, but rather are different ways of viewing the actions and behaviours of sets of people.

I’ve been developing my tools as a way of allowing the whole cohort of medical students to share resources, but I’ve been mentally modelling them as a group in Dron & Anderson’s definition - a set of people who know each other socially, and for whom the online social tools are a side-channel for other modes of interpersonal interaction. I’ve been think of ways of making the network explicit, but haven’t done much beyond exposing their class groupings and allowing commenting and rating of each others’ resources. But my software design is really for collective software - software that succeeds though its emergent properties - in my case, the collaborative group sharing and rating is intended to allow students to collectively discover the best learning resources.

But my interviews with students are making it clear that what I’m designing isn’t what they’re really looking for (it’s a small sample so far, but the message seems coherent). They are working at the group level - in this case, in the small problem-based learning tutorials groups that they are learning in. They are creating Facebook and other groups to support these small learning groups, and the collective behaviour of the cohort isn’t of real interest to them. Even the network itself isn’t of real consideration; their actual social network is embedded in Facebook, and there isn’t really to a large degree a separate academic social network beyond that created through the course structure of PBL groups and Clinical School groups.

What does this mean for my research and development? Firstly, I need to focus on the real environment that the students are working in, and create affordances that suit that environment. For example, creating small group sharing spaces will be of more value, and allow these groups to share resources with each other without worrying about the wider cohort judging them or freeloading off their hard work. Secondly, I need to keep a sharp focus on the functionality I’m adding, explicitly noting what level of social unit I’m targeting. This means adding a tool to track, for each feature I add, what social level it is addressing (and if it meaningfully addresses any level).

Profile pages

I've been delving further into the students' use of their profile pages in Compass, and am finding some interesting, if unsurprising patterns.

1. Students are mainly interested in their profile pages when they start the course. 

Profile edits, by action type


Profile edits, by cohort
 

The reason that the spike is so much lower at the start of 2014 then it was at the start of 2013, was that staff uploaded the students' photos for them in 2014, whereas students uploaded their own photos in 2013 (I have excluded staff activity from the graph). The spike was expected, but it interesting to see the students still checking their photos and uploading new ones throughout the year. It'll be interesting to check how many students have modified their photos and how many have multiple photos available online.

2. Students are mainly interested in each others' photos when they get assigned to new groups.

Profile page views by Cohort 2013, per week

There is still an average of 114 profile page views per week outside the peak times, but there are very clear peaks (above 1000 views/week) at the start of each school year.

3. Students are mainly interested the profiles of student in their own cohort

The x-axis lists all the student profiles, ordered by number of page views; the y-axis is number of page views

Particularly during the first weeks of each year, students are mainly looking at the profiles of students in their own cohort. The above graph shows each cohort has a large number of page views of around 300 students (their own cohort), with a smaller number of views across the other cohorts.