Saturday, 15 August 2015

My own personal university league table (2016)

In the past I have published a master league table based on a conglomeration of national and international tables. To my reckoning no one has made a master table that has as many references as mine did (10 tables in all), however this time I have finally made a league table of my own.

  Introduction 


 This league table is based on the 2015 clearing vacancies. In essence the methodology is simplicity itself; simply divide the number of vacancies by the total number of courses available. However in practice it proved more difficult, data collection does take some time, and despite the fact that most of it was done at 1am in the morning (when hopefully neither UCAS or the Universities were accepting clearing offers) this cannot be guaranteed, so there is a possibility that there is some alphabetical bias going on.

Motivation


League tables are a controversial subject by nature. My view is that people on both sides of the debate (particularly the anti league table position) can over exaggerate both the benefits and the shortcomings. League tables are not useless, they give a quantitative measure of (mostly) undergraduate performance of the institution. Right from the get go this is always going to be superior to a vague qualitative notion of prestige if only because the position of a university is determined by empirical data rather than hearsay (even if the methodology of league tables is shoddy, at least it exists to criticize!). On the other hand, students should not be overly concerned about whether their institution is 10th or 20th on any given table (or even my master table!), other factors such as the individual course and whether or not a student likes the ethos and atmosphere of the institution is also very important. Use league tables as a guide; do not dismiss them but do not treat them as gospel and ideally read the methodology of each table and decide for yourself whether the ranking is a good proxy for what you are looking for in your university. With all this in mind one can see the use of a ranking based on clearing places: Universities with a lower proportion of vacancies are more attractive to prospective students and thus in their eyes have higher prestige (one would hope this is due to merits of the university). Of course it isn't a perfect system, universities may have a large number of clearing places because they want to poach students with very high grades after results day (although overachieving students do have the option of using a separate system called 'adjustment' as an alternative to clearing) or simply have a weakness in a particular subject area which is not representative of the university as a whole. Therefore it might be better to think of this as a 'prestige' rating (albeit a very imperfect one) than a league table, in any case the vast majority of league tables use more than one criteria (indeed this 'league table' wouldn't meet the basic requirements necessary to be included in my own master table).

  Methodology


The UCAS clearing search function was used on the 15th of August to search by institution for the number of courses available in 2015 clearing, and total courses available in 2015. This was primarily done at ~2-3am UK time for two reasons: firstly any processing done by either UCAS or the universities themselves which could result in a time dependence of the listings was minimized, and secondly to request the website at a time when there would be low traffic from students who actually require use of the service. Unfortunately some universities were so efficient that most clearing places had already gone by that time; in order to take this into account information regarding Russel group Universities from the Daily Telegraph was used to provide supplementary information about the number of places available shortly after results day (on the 12th). While the ranking is based almost entirely on the search conducted on the 15th, some slight adjustments in rankings were performed based on this supplementary data when the discrepancy was very large. The individual adjustments are listed below along with some other outstanding anomalies. If the exception is not listed here, assume any discrepancy was not considered significant enough to justify messing with the rankings based on data collected on the 15th.

 Health warning Regarding the supplementary data 

Rather weirdly there is a discrepancy between the total number of courses offered given through UCAS, the daily telegraph clearing search function, and the Russel group places available listed by the daily telegraph on the 12th. Bizarrely sometimes the number of clearing places offered actually exceeded the total number of courses available by UCAS. The discrepancy cannot be accounted for by controlling for only Undergraduate degrees, full time degrees, or single subjects only. As a result a weighting based on the total number of courses in UCAS could be used to roughly compare the Russel group universities (with clearing places), however any cross comparison between Russel group and non Russel group universities cannot be justified, therefore adjustments in the ranking have only been made in the event that the Russel group only ranking is out of order AND the change in places between the 12th and 15th would have to have been significant.

Exceptions 

Surrey: 
Surrey had zero clearing places available on the 15th, and is not a Russel group university (the only non-Russel group institution apart from the filled St Andrews to be so high in the table) so I have no information available about exactly how many courses were open on results day. However, in 2013 there were about 40 courses available which corresponds to about 85% of courses unavailable. Assuming this figure puts Surrey in the top 15 anyway, but since there is no usable data from 2015 I have kept surrey in 8th place, just below the other universities with no clearing spaces.

KCL, Loughborough and Birmingham:
These three universities are close in the table to begin with anyway. The supplementary data indicates that Birmingham should probably be just above KCL, but its questionable whether this is significant or not. On balance these three institutions (with Loughborough in the middle originally) cannot be differentiated and so were given the same ranking.

Southhampton:
Southampton had a lot of clearing places available on results day. When compared directly with the other Russel group university Sheffield (a university with a similar number of available courses) it is very hard to justify putting Southampton so far above Sheffield (much less 21 places). On results day Southampton probably had a higher proportion of courses available than Sheffield so it is placed one place below Sheffield. Unfortunately this placement is completely arbitrary with respect to non-Russel group institutions and one could argue that Southampton should be rewarded for filling its places up quickly. This is definitely the university that highlights the weakness of this ranking system the most.

Exeter: 
Bizarrely UCAS did not list the correct total number of courses for this university in 2015, so a 2016 search had to be used. This has not affected the ranking of this institution.

Miscellaneous

Universities are ordered alphabetically in the event of a tie. Note that only universities that appeared in the 2016 Complete University Guide also appear here, there is every possibility that small specialist institutions would otherwise do very well in this ranking were they to be listed here.


General health warning

Please do not take the positions of universities here too seriously, or base important judgments on this table. This should primarily be for interest purposes; how people interpret the results here in terms of 'quantitative prestige' is up for debate. Keep in mind also, that this is a single-criterion league table. In general league tables use more than one criteria (and typically more than five). Please take the positions here with a rather large pinch of salt.

  Results

The research intensive Russel group universities tend to (colourd yellow), perhaps unsurprisingly, be very popular choices among students and cluster towards the top of the table with few (if any) vacancies. The only university that did not enter clearing, and is also not in the Russel group is St Andrews (a member of the disbanded 1994 group). 

115 Liverpool Hope 0.6
116 Derby 0.6
117 Aberystwyth 0.6
118 Sussex 0.5
119 Trinity Saint David 0.4
120 Sunderland 0.1
121  Harper Adams 0.0
122 Newman 0.0
123 Nottingham trent 0.0
124 Royal Agicultural  0.0
125 Royal Holloway 0.0
126 St Mark and St John 0.0


The distribution of results is rather interesting, appearing to be bi-modal in nature (see histogram below). In real terms this implies there are a significant number of institutions that are both oversubscribed and under-subscribed, and relatively few that have a roughly equal number of filled and unfilled courses. For the universities with many vacancies (left of histogram), the current system does not seem ideal with students essentially circumnavigating the application system. Those with the vast majority of courses filled, may simply not need clearing because the universities are so popular with prospective students. 

























In terms of how this proxy compares with a multi criterion league table, below a snapshot from the complete university guide (2016) is shown:


In the top 10 rankings the same 7 universities appear, and in the top 20 rankings the same 14 universities appear (and all the top 10 of my rankings appear in the top 20 of CUG). For universities that rank lower in the table, however, there is a greater divergence between the tables. Of course the clearing table is unable to distinguish between the top 7 universities.  


  References and Acknowledgements 


1 (http://search.ucas.com/): In finding available clearing places (2015) and total number of courses for each university (2015)
2 (http://surrey.ac.uk/clearing and its cache from 2013 https://web.archive.org/web/20130814062109/http://surrey.ac.uk/clearing): Clearing information from Surrey University
3 (http://www.thecompleteuniversityguide.co.uk/league-tables/): The same universities hold positions in this table as in the complete university guide. 
4 (http://clearing.telegraph.co.uk/InstitutionSearch.aspx): Telegraph clearing search
5 ( http://www.telegraph.co.uk/education/educationpicturegalleries/11798242/Russell-Group-universities-in-Clearing-2015.html): Russel group universities in clearing on results day.


Terms of Use: Please link back to this post if you wish to quote any of the text here or results from the league table. Everything written here is in my own words. Please do not use the information here to make any important decisions regarding applications, this is for interest purposes only. 


No comments:

Post a Comment