10 January 2014 Leave a comment
I feel a bit embarrassed having only secured one single blog post in 2013 . My mentor Martin Weller would be ashamed of me. Interestingly though, according to my annual feedback, my blog received almost as many visits in 2013 as 2012 (around 3000 visitors). Is that a reflection that the content is becoming more valuable as time passes?
Reflecting on 2013, for me it’s been the year of turning aspirations into products. For the iSpot project for example I created what I called a “Technical Roadmap”, which is really a grand way of saying that we had so much to deliver from a total of four different funders, we also been involved in BBC TV series (The Great British Year) and in the OU’s first Futurelearn MOOC on Ecosystems. (Which I took part but sadly became a drop-out!)
As a consequence we needed to ramp up the technical management of the project for what was an extremely challenging year and the Technical Roadmap helped us to keep our sanity (most of the time). Richard Greenwood has created a project blog about the main technical work during 2013.
Here are a some of my highlights:
1. Internationalisation/Community (the link takes you to the UK and Ireland community)- This is by far the biggest technical feat of the year for iSpot. The system now supports numerous communities organised according to geographical or taxonomic criteria. Richard Greenwood worked very hard on the functionality, which uses polygon mapping to calculate areas (and use multiple polygons so a region such as the UK, or Eastern Europe can be mapped out). The difficulty was providing communities without destroying the taxonomy (species dictionaries) as these sometimes span many areas. With the UK is was simple but now there are multiple dictionaries (one for Global iSpot) that need to be used in the correct places. Richard therefore couples the taxonomies to the observations locations, but decoupled it from the community (polygon) model, thus allowing freedom to create communities without having to use a dictionary that wasn’t relevant to their locale. The technology used is MariaDB and Open Street Map for creating polygons (and Google maps for displaying them). Richard also implemented Geo-IP to direct people to the correct community be default and the system will also allow people to move to different communities. Communities don’t have to be countries (we now have a budding Chilean community on iSpot for example ). Communities have their own News items and maps which are centred on their geographical region, and observations relevant to that community. Communities don’t just have to be geographical, they can also be around organisations or species or in fact anything that can be filtered against within iSpot, this makes the feature potentially very powerful.
2. A species surfer – The species surfer (or ID tool as it was originally called) allows anyone on iSpot to browse the species dictionary (taxonomy) using images to represent the main categories and sub-categories. Within a sub-category people can look at the variety of types to track down ones that are similar to their own observations. We know from talking to users that this is something they’ve been interested in having. Many people use Google and other sites to try to find out more about their observations and we thought that since iSpot has over 250,000 observations, the majority of which have been accurately identified, we should use that feature and draw it to people’s attention. It also acts as a learning tool and we hope it will be useful for field studies and research, from novices through to experts. This has only just been released so we still have further work to do to improve it but we want to get feedback from users since we know that there is still more work to do on this. The iSpot team have provided help information to guide people in how to use it correctly.
3. Intelligent quiz – The existing crowdsourced identification model within iSpot, rewarding improvement in ability to identify observations, provides some of evidence that people are learning and improving their understanding of nature through iSpot, however it isn’t full-proof. For example a person may gain reputation through identifying very common species and without expanding their knowledge of other species. We therefore require empirical evidence of improvement in people’s ability to identify a greater variety of observations as their reputation improves; the iSpot intelligent quiz is designed to test this knowledge. The quiz was launched in July 2013, since then around 350 people per week have taken one or more quizzes, so an average of around 50 people per day. The quiz is tailored to the level and subject area that people request when they start a new quiz on iSpot. The reputation level that iSpot provides is a good indicator of the level that people should take but there is no restriction on the level so, for example, a level five expert could take a level 1 quiz and vice versa. The data from the weekly logs shows however the people are averaging about 7 out of ten for quizzes across the skills levels which suggests that people are naturally finding a level which challenges them.
The quiz has a number of different types of question that test a range of knowledge within a specific domain, some questions are multiple choice and others are about entering the correct name or type of observation, some examples are shown below:-
The quiz is largely image-based and relies on people correctly identifying observations. The quiz is open to both visitors to the website who have not yet registered, and also to registered users. Registered users have the benefit of being able to look back at previous quizzes they have taken to compare results. As part of the intelligence the quiz tries to select images which have been agreements and ones which are non-contentious, for example it will attempt to filter out hybrid types. In the example below people can use the button in the right hand corner of the image to expand it and see additional detail.
Certain questions prompt people to enter correct names associated with an image, they are based on the names given within the species dictionary on iSpot. The system will look up the dictionary and offer suggestions for entries that match, or which are very similar to, the name entered by the user.
We collect overview information about the quizzes on a weekly basis, including information about preferred groups, as you can see from the chart below birds consistently prove to be the most popular category for people taking the quiz.
The weekly statistics show us that the percentage of visitors who take quizzes compared to registered users varies from week to week.
For example during w/c 16th September 2013 about three quarters of people taking the quiz are registered users as indicated in the following diagram.
Interestingly during the previous week the ratio was more like 60/40 in favour of registered users so this seems to be indicating that as time passes the quiz may be becoming more popular with registered users however this will require further data analysis.
Each quiz has up to ten questions so the table below shows that during the previous week there is an 80.7% completion rate.
The completion rate for the previous week was 89.1% and completion rates seem to fall consistently within 80%-89% percent range.
We are tracking the average scores of people who take the quiz and the results show us that there is only a very slight variation in score between people who class themselves as novice and take a level 1 quiz and people who class themselves as expert and take the level 5 quiz.
There is a slight decrease from 7.5 to 6.5 going from level 2 to level 3 and beyond however it is worth bearing in mind that the quiz provides novice users with up to three “lifelines” to use to help them (a lifeline is typically where two of the four choices are removed to make it simpler for people to find the correct remaining answer).
We have yet to analyse the raw data coming from the quizzes and because the service is relatively new we need more time before we can start to get useful trend data to help us demonstrate that people are increasing in their knowledge of nature through using iSpot.
In particular we need to understand the relationship between the amount of time people have been using iSpot and the level of knowledge they have attained. The data already indicates that people who use iSpot are gaining knowledge about nature and over the next few months we will be conducting further data analysis to understand exactly how this is being achieved.
These are just a selection of some of the new features in iSpot (I have at least 24 more to share with you!). I am very interested in how these systems evolve over time and the nature of the co-evolution of the technology and the people using that technology.
The Facebook we see today is very different from the first iteration of Facebook.
People are generally much more technology aware, and use technologies frequently for “selfies” and to share with others in a connected way. Systems must therefore evolve to support the changing perceptions of users to technology and iSpot can naturally support learning using images and photographs that people nowadays naturally want to share.
I’ve summarised some of the latest iSpot features that explain this co-evolution process in a presentation that I gave in December. We “technocrats” rely heavily on the community, and the subject experts to help us create services that are useful and provide mechanisms of learning and improvement.
I will be continuing over the coming months to give examples of the richness of the systems that we’re working on the Institute of Educational Technology. Working in partnership with the Science Faculty and Open Media Unit and the 36,000 users of iSpot.