Installing Unity Remote

I found setting up the Unity remote is easy to do on Android, I expect that the iOS version is a similar easy process. The Unity Remote is found on the play store, there are a couple on there so make sure you get Unity Remote 4, it should be free and developed by Unity Technologies. The play store doesn’t have an approval process and while this is great for developing and deploying apps fast it does has a problem with fake apps, always be carefully when downloading on the play store!

Read more ›

Posted in Technology Tagged with: , , , , ,

Visual breakdown of categories on wordpress blogs using R

This is a very simple recipe, just a few lines to get an indication of tags/categories being used on a WordPress site. The idea is that I use R to read the RSS feed of the blog, pick out the tags and categories and display a pie chart of the tags being used. Since tags and categories in wordpress are set by the user it gives you an indication of what subjects the authors think their posts fit in to. It might be a useful script in keyword planning and seo, but I just use it to see how my interests are changing.

Read more ›

Posted in Data Analytics

Are we too naive in education?

In my last post I had been thinking about technologies that online communities use and how the technology pushes people to communicate in a certain way. I pushed this out to my Google+ feed, perhaps because Google use plus metrics to an rank article but I would like to think more so because I have friends on there whose feedback I value. When my Google notification button told me I had a comment from Sheila MacNeill I was pleased, not solely because comments will get me some points in Google magic ranking algorithm but more importantly Sheila has a knack for reading through a wall of text, pulling out the important stuff and translating it in to concrete questions. This was Sheila’s comment after reading my post:

Read more ›

Posted in Data Analytics Tagged with: , ,

[slideshow][mlg] 46 Learning Analytics Gifs only EdTech People Will Understand

I’m sorry there are no gifs, I understand if you leave now but it will hurt my ‘time on page’ ratings. Please stay a while and pretend to be interested.

Writing what I truly think about a subject on a blog is a difficult process. The first hurdle is the title of the post, which will find itself in the title and heading tags on this page and consequently be used by Google to decide what this page is about and where it should appear in rankings. I get 75% of my traffic from Google so I better make sure I get some juicy up and coming keywords in the title. I also need to make people want to click my link on social media, and the $850 valuation of Buzzfeed tells me lists and reaction gifs is the way to go. After the title I have to think about images for pinterest, keywords for Google, hyperlinks for HITS style algorithms and how this will affect my chances of landing a new job…

Read more ›

Posted in Education Tagged with: , ,

Ric Flair Promotion Jumps

I posted an experiment on a wrestling subreddit a few days where I used DBpedia to explore the links between wrestling stables/ .To my surprise wrestling fans are really interested in the things you can find out using structured data from wikipedia. I was contacted by a graphic  designer asking if I wanted to collaborate on doing some wrestling visualisations together. I thought it would be a great idea and we have both gone away to think about visualisation ideas. Mining the following data was not my idea, but the idea of a fan on reddit, these are some experiments that I have made getting that data ready for a visualisation for this. I’ve found this a really interesting project to work on because pro wrestling databases seem to be very tightly guarded!

The idea was to grab the different promotions that wrestlers had worked for and visualise them. I couldn’t find a decent data set with this in, but have found a site that lists all a wrestlers matches over multiple pages. Here is an example for Ric Flair. I decided that the way I would decide which promotion a wrestler worked for was by using the last date a wrestler worked for a promotion on TV. That means if Ric wrestled for the WWE on 1st August and then was next seen on TV for WCW on 1st of September, I would create a visualisation where Ric worked for WWE throughout August. I’ve gone for this because I think I can work with the data that way and because I think it fits in with the reasons why the community would like this to be visualised. I’ll have to ask the fans later if this is correct or not!

First I scrapped all the results in to R using the following:

Not pretty, but it works.

I then realised the promotion field was blank! This was because the webpage uses images for this field. I noticed that promotion names were in the match Title and created my own using grep on likes something like this:

I wrote this to csv:

I then deleted what I didn’t want by hand and bunged the result in Google charts. You can see the interactive version here.

There are a few things I’m not happy about. I did a little in hand in CSV which makes it prone to my lazy mistakes. I’m also not sure that TV events were the best way to do things. I’m also missing my labels and the such, its a start.

Ric Flair jumps


Posted in Data Analytics Tagged with: , ,

Scraping a HTML table into an R dataframe

For some work I plan on doing for the LACE tech focus blog I wanted to get some information off a webpage and in to an R dataframe. It turns out this is a three line solution (1 line if you throw your URL straight to the function paramaters and have the XML package already installed).

I’m pretty sure this is something I will need later in life.. thought I better blog it somewhere..

Posted in Data Analytics Tagged with:

Update on Wrestling visualisations

A few days ago I started playing with dbpedia data. I had spent a while trying to find interesting people story lines to find information about. I had toyed with story lines in TV soaps and video games, but after a decision with a friend where I was introduced to the concept of a wrestling stable I decided to explore links between factions/team mates in the pro wrestling world. I decided to crawl through all the links between all the stables in dbpedia and and play with them in Gephi.

A few hours later I had made some Gephi style pictures, but realised I knew nothing about pro wrestling and needed some input from people who do. I headed over to Reddit’s wrestling forum /r/squaredcircle and asked them for feedback on my post. I was really taken back by how helpful an online community can be. I had lots of positive comments and interesting feedback that helped me think through the dataset. It really made me think about telling ‘stories with data’ (I hate that phrase!) and this whole thing is really a collaborative thing. Some great discussions later knew much more about the dataset. Aside from some interesting wrestling storylines I also picked up some things about mining dbpedia.

Data is frozen at June 2013, some members of the community pointed out that I had missed newer groups. Also the data is not consistent. I had chosen to use the little piece of information on a Wikipedia page that says “Member of” or “Former Member of”. This is actually pretty limited as people don’t seem to fill in these all the time. I could mine categories or something. I’m not sure there is a best way to go round this. I guess it depends on what you are mining.

For example I used the following query at dbpedia to grab members of stables:


Some Redditors spotted major wrestlers missing, wrestlers such as Ric Flair! Looking up Ric Flair  made me realise he hasn’t got a rdf:type dbpedia-owl:Wrestler, as such so he wouldn’t appear in my diagram. The addition of someone as influential as Rik Flair will seriously change the diagram I’m sure and it was a shame he was missed. I notice that most wrestlers have the word wrestler in their description, so I could get them using this instead


To get the best of both worlds I can use DISTINCT and UNION in my query to grab one or the other. There is no guarantee that I won’t miss a wrestler still because they 1) Might not either 2) Might not even have been added to Wikipedia, or be in the right ‘Member’ or ‘Former Member fields’

Next Steps

I’m not going to carry on with the visualisation that I made because dbpedia just has too many gotcha’s in the dataset. I did however have some offers to collaborate and I think I’m going to follow up on these and try and pick up some new skills from my new found friends.

While the data in DBpedia might not be perfect the one thing I am really sure about now is that it is a great way to start a discussion about something with a community! Thanks again to the members of /r/squaredcircle.

Posted in Data Analytics

Exploring Wrestling Stables

In pro wrestling a stable is a group of  wrestlers who form an alliance and wrestle together, they help each other out the way typically team mates would, for example by sneaking sledgehammers in to the ring when the referees have their backed turned. When I was younger my favorite stable were called D-Generation X – a bunch of loud mouthed rebels who broke the rules and defied all authority. Recently a childhood friend told me that one of these rebels was doing a talk in our local town with some friends and was charging £50 (!) a ticket. To go to an event in our town £50 is quite a lot of money, usually the top bands will charge £50 at an arena, you would probably get change for £50 if you went to see Metallica at a football stadium. So wondering why these performers from the  90s get to charge such a fee I looked the guy up on Wikipedia, it turns out that the performer, real name Sean Waltman had been in lots of stables and a little more research reveals that wrestling fans LOVE stables. Knowing exactly who has been in a team with seems to be part of the fun of watching wrestling and going to a live show to hear wrestlers tell stories about previous stable mates commands a higher ticket price than going to an actual wrestling event itself.

I found this fascinating and wanted to explore who Sean Waltman had been team mates with. In fact I wanted to map out ALL the wrestling stables that ever existed. I noticed on Wikipedia that there are pages dedicated to stables and these stables have entries for members and previous members.

A little data mining later and I had a list of all the stables in wrestling and who belonged to them. I mapped this out in a visualisation and created two massive maps of wrestlers. It turns out there are a lot of stables in pro wrestling. I’m thinking of digging deeper in to this but I’m not sure what I am looking for, I might ask seek out the wrestling fans of Reddit for help.

The Connection Maps

The maps are huge. About 7mb each and 3600×3600. If I click the images below in firefox it shows me a zoomed out version and then I can click it to explore it. I know they are not great at the moment because there is to much information in there, but I think my next step is to go and ask wrestling fans what sort of thing they might be interested in finding out from this data.

You do need to click the image which will take you to imgur, then you need to click it again to get to the image itself and a final click to zoom in to explore. The nodes represent people and the lines represent the fact they were in a stable together.

A community detection algorithm called the ‘Louvain modularity method’ has been applied to it, this tries to detect communities of people within the network. Each person has been color coded according to the network they are in.

The dots have been resized too, I have resized them according to their ‘Betweenness centrality’ measure. Really simply explained, this is a score that is found out by the number of shortest paths between other wrestlers that go through a wrestler. So for example, in the map the quickest way to get from Darren Young to Adam Birch is through CM Punk so CM Punk gets a point towards his Betweeness Centrality and his dot gets bigger. For a proper and better explanation you are better off reading the Betweeness Centrality Wikipedia page.

I am also aware I have left off important information from the graph! I need my wrist slapping for missing out what they weighted by and the such.




Updates from wrestling fans of Reddit

  • So the people at squaredcircle have the comments coming in. I’ve created a quick video because I find that it helps me to get my head around things and I’m uploading that at the moment. u/lykki suggested I try doing a ‘Prezi’ using the image, I think that might be a good way to deliver the information in chunks.
  • Also it was pointed out that my images are JPEG.  I exported them in PNG, but it turns out IMGUR has been converting them; I’ll try to find out why.
  • I can use the d3 tree layout to make a history of who trained who. Might be a good idea to use a sankey diagram thingy.
  • Some interest in representing this in d3 style network graphs
  • I’m missing Rik Flair! Best check my dataset to work out why.
  • I am completely blown away by how helpful a community can be, a follow up post on how helpful the community was might be interesting
  • /u/artcarden has 3 really good points that I have copied here. I never knew stables and factions were different:
  1. There’s a debate about the differences between stables and factions; for your purposes, you could probably consider them the same thing.
  2. I like including tag teams, but I think you’ll want to make sure it was a recognized team rather than a one-off or month-long deal (Rybaxel counts, Cena tagging with Roman Reigns a few times doesn’t unless they combine to form the Super-Powers).
  3. A next step: calculate each performer’s “Flair Number” or “Hogan Number,” like a scholar’s Erdos Number. People who teamed directly with Flair in the Horsemen or Evolution would be Flair 1. Assuming there’s no direct Flair/Waltman link, Sean Waltman would be Flair 2 because he was in DX with Triple H, who was in Evolution with Flair. Edge would be Flair 2; he was in Rated RKO with Randy Orton, who was in Evolution with Flair.

Final Update

After some great discussions with the guys at squaredcircle it turns out that there are quite a few bits and bobs missing. I expected this because well, it’s wikipedia so it can only really give you a representation as good as the data that goes in it. There are a few things I could do to make it a little more accurate but it doesn’t seem worth the tweaking on this project. I’ve had a few offers of visualisation collaboration and it seems to make more sense to start again with those people who have more knowledge on the subject.

So I guess I’m parking this particular project and taking up some others with some fans I’ve met on Reddit. I’ll write up my experiances, share it and start on something new. Here is the R script I used incase anybody finds it useful. I realise I’ve done the whole thing in 3 steps instead of 1 but that was for debugging reasons and now it is never going to get finished:



Posted in Data Analytics

Getting information out of Google Scholar

The Google Scholar site doesn’t have an API, which is a shame and has left me to park one of my current projects on the sideline for now. I still spent half a day working out the best method to get information out of it so thought I would write up what I found in case it was useful to anyone else. The particular project I was working on was grabbing citations of papers and if anybody is interested it is parked because not all papers have a Cluster ID, which I naively assumed they would. It doesn’t seem worth going back and finding a work around as I’ve been down this route before trying to scrape things from websites only to find that they break after a UI tweak.

For those who still want a go at poking Google Scholar I found a python script called, written by Christian Kreibich worked very well and can be accessed from Github here. I found this forked repository by  Korbinian Riedhammer also adds the option to grab citations based on the Cluster ID.

It is easy to get started, on MAC OS X I went with

  1. sudo easy_install BeautifulSoup
  2. Download script
  3. Try something like: -c 1 –author “D Sherlock” –phrase “tools for online Habits”
  4. or  Or –cites –cluster-id 13746912682491308133 for citations


Posted in Technology Tagged with: ,

The death of Free2Play beyond video games

Way back in February the EU commission raised concerns over business models in the mobile phone ‘app industry’. There were 4 major points of concern, these are taken directly from the EC press release :

• Games advertised as “free” should not mislead consumers about the true costs involved;
• Games should not contain direct exhortations to children to buy items in a game or to persuade an adult to buy items for them;
• Consumers should be adequately informed about the payment arrangements and purchases should not be debited through default settings without consumers’ explicit consent;
• Traders should provide an email address so that consumers can contact them in case of queries or complaints.

Two weeks ago a follow up release gave details of changes to the Google Play store that will take place before the end of September:

These include not using the word “free” at all when games contain in-app purchases, developing targeted guidelines for its app developers to prevent direct exhortation to children as defined under EU law and time-framed measures to help monitor apparent breaches of EU consumer laws. It has also adapted its default settings, so that payments are authorized prior to every in-app purchase, unless the consumer actively chooses to modify these settings.

For those who don’t play games this might be a little but confusing. The idea is that currently Google (and I presume Apple) have games that can be downloaded without a cost, but these games still make money for their publishers. Looking at the Google ‘top grossing’ section on the play store reveals that only 1 in the top 50 grossing games is actually paid for (Minecraft). The games that can be download without payment often label themselves as free and the games industry dubs them Free2Play. Although when 49 out of the top 50 Google Play grossing apps are Free2Play it is clear they not only make money somehow, but are actually pretty good at making it. The techniques these games use to make money is through what is called ‘in-app purchasing’. The idea is that instead of paying for a game you pay for items in the game, the game publishers claim you don’t need the items to play; hence Free2Play.

None of the top Google Play games have a price, they are all free*

None of the top Google Play games have a price, they are all free*

This business model requires developers to design their games a little differently than they used to. In the older, now dubbed ‘Pay once; Play forever’ model there was a set amount of money that a publisher/developer would get from a purchase of their game. In the Free2Play model the developer has the task of tempting the player to buy more in the game, and they do this buy leveraging peoples natural desire for achievement, those that pay get further quicker than those that don’t. Another way to tempt players us to dig into the desire to compete and beat their friends, a closer look at the Google play store shows that the top games are all linked to social media sites such as Facebook. Those that have played Candy Crush might familiar with the game map those shows which level friends are at and what score they got, it does not you how much they paid for items to produce that score, giving those that pay the advantage in their social network.

Players find themselves in a game that is pulling on natural desires to compete and gain status. The only way to win this game is to pay and the EU want to make it clear that when you pay and what for. You could say that the game techniques “strive to leverage people’s natural desires for socializing, learning, mastery, competition, achievement, status, self-expression, altruism, or closure.” Which funnily enough is the exact sentence used to describe Gamification at Wikipedia.

The games industry is young has moved at an extraordinary pace, the technology and business models that fuel it changing radically many times since the first video game console was connected to a TV some 40 years ago. You could say that this the quick pace has given both industry veterans and consumers a keen eye for social problems rising from these changes. The push back to Free2Play initally came both from industry veteran’s such as Ian Bogost’s work in what he calls his work on ‘Exploitationware’ and from consumers alike and it feels good that something is being done.

It doesn’t quite feel this way in other industries. A few months ago I got some free Facebook advertising credit, and while I was sure this was a way for Facebook to show me I could spend money with them to ‘leverage my natural desires for socializing, learning, mastery..’. I wanted to experiment with it to see what exactly they were doing. I decided not to write the results up at the time as the whole experience made me feel pretty sick and I decided it would be best to reflect and leave the write up for later. What I can say now is that the experience did make me realise that while Facebook was free to sign up and use  the end user is basically the product, being used to generate data for Facebook, sell adverts to or to be a unwilling research participant. The same is true for Twitter and the like. Instead of an EU inquiry to techniques employed by these industries we can say something along the lines of ‘that’s the sacrifice I make to see pictures of my grandkids’ or ’I’m happy to let Google know where I am 24/7 so my phone can constantly tell me the nearest place for real ale’.

We find ourselves playing similar games in education, and as a gamer I find it somewhat bizarre that the term gamification is often used as a positive thing in the education world; as if underhand techniques pulling on desires of desperate students to hit your key performance indicators is a good thing. I feel the techniques are somewhat worse in education then gaming as the things we try and get them to do are more complex than that of the ‘share and pay money’ the game industry wants. In the game industry the deal seems reasonably straightforward, I want to be entertained and compete with my friends, in return I will pay for the upper hand. But what is the deal to students who find themselves trapped in the games their education institution is playing with them? Our system says the best students are the ones that pay £9000 a year and play the game exactly as we say and institutions are telling them to play the game in a way that improves the institutions prospects and not always the students. The well known example being the student feedback form where students were implicitly told that negative feedback would effect their chances of getting employed, what can they do?

I think perhaps the games we find ourselves in with social media companies and education are somewhat entwined and not easy to pick apart.

The stakes at risk are much covert than those in the app industry are much more dangerous. My hope is that a serious look at the death of free2play may expose some of the morals behind striving to leverage people’s natural desires for socializing, learning, mastery, competition, achievement, status, self-expression, altruism, or closure.

Posted in Computer Games, Education Tagged with: , , , ,