Wednesday, 28 November 2012

Widgets, Web Services and Libraries - Oh My!

The time has come to fulfill my promise [0] to write a blog post on our use of widgets at Swansea University.


To cut a long story short, the decision to focus on developing widgets came about through the realisation that the Web Team were being asked increasingly to provide data to a number of disparate university web based systems such as Blackboard, iFind Discover (Our library catalogue powered by VuFind and Voyager) iFind Research (Our remote discovery service powered by Xerxes and Metalib / SFX) and our intranet and web pages. We therefore wanted to create a service which followed a recognised pattern, required very little technical proficiency to implement on the client side and was easy to share and distribute. Thankfully, our lead developer Geraint always has his finger on the pulse so he was able to suggest that widgets, self-contained, platform agnostic web elements, would meet all our needs, allowing us to deliver content, style and functionality in one easy to access bundle.

The fundamental basis of approach was taken form an article by Alex Marandon [1]. In our environment, this equates to something like:

1) Data API (e.g. Rebus List - our reading list solution from PTFS Europe)
2) Data Access Object (Web Team Developed object which queries the API)
3) Widget Controller (Loads the Data Access Object, processes data, returns data in required format)
4) Widget JavaScript (Collects variables, sends them to the controller via ajax and uses a callback function to process the results)
5) Client added script tag and target html element:

<script type="text/javascript" src=""></script>
<div class="widgety-goodness" data-id="1234"></div>

This method of delivering content places virtually all of the development requirements for a service on the web team. All a client needs to be able to do is embed two lines of code in their web page with any required variables. The only further developments a client may wish to undertake is to create custom css styles or JavaScript / jQuery behaviours.

A Guided Example

For those more interested in the technical details of the service, I will now try to outline the development of our Rebus List Widget with a particular emphasis on modifications we have made to Alex Marandon's original post.


Get a particular course reading list from Rebus List by supplying a valid course code


1) Script tag and HTML element (Blackboard Team - Swansea University)

The Blackboard team embed the required script and html elements to each course page and generate the course id. They have also created their own style sheet which will override the default css supplied by the widget.


<script type="text/javascript" src=""></script>
<div class="rebus-course-reading-list-widget" data-course="CS-061">&nbsp;</div>

2) Widget JavaScript (Web Team - Swansea University)

The Widget JavaScript grabs the course id from the html element attribute and submits it via Ajax to the Widget controller. If successful, it loads the returned content in to the html element in addition to grabbing required resources like html5shiv [3] or css files.

The major differences in our JavaScript files compared to those suggested is that we have formatted them using JSLint [4] and added methods to deal with problems caused by the way IE7 loads css and js files.

This particular example does not have any jQuery actions associated with the delivered content but it could be added as part of the $.getJSON method if required.


/*jslint browser: true, devel: true, sloppy: true */

(function () {
    var jQuery,
        main = function () {

            jQuery(function ($) {

                var $targets = $('.rebus-course-reading-list-widget'),
                    devEnv ='env=dev') !== -1,
                    baseUrl = devEnv ? '/widgets' : '//',
                    styleSheet = baseUrl + '/css/rebus/course-reading-list.css';

                    ? document.createStyleSheet(styleSheet)
                    : $('head').append($('<link>', {
                        rel:  'stylesheet',
                        type: 'text/css',
                        href: styleSheet
                // Do we need html5shiv?
                if ($.browser.msie && $.browser.version < 8) {
                    script_tag = document.createElement('script');
                    script_tag.setAttribute("type", "text/javascript");
                    script_tag.setAttribute("src",  baseUrl + "widgets/vendor/html5shiv/html5shiv.js");
                    (document.getElementsByTagName("head")[0] || document.documentElement).appendChild(script_tag);

                $targets.each(function () {
                    var $this = $(this),
                        course = $this.attr('data-course'),
                        uri = baseUrl + '/controllers/rebus/course-reading-list.php?callback=?&course=' + course;

                    $.getJSON(uri, function (data) {

        scriptLoadHandler = function () {
            jQuery = window.jQuery.noConflict(true);

    if (window.jQuery === undefined || window.jQuery.fn.jquery !== '1.5.2') {
        script_tag = document.createElement('script');
        script_tag.setAttribute("type", "text/javascript");
        script_tag.setAttribute("src", "//");

        if (script_tag.readyState) { // old-IE
            script_tag.onreadystatechange = function () {
                if (this.readyState === 'complete' || this.readyState === 'loaded') {
        } else {
            script_tag.onload = scriptLoadHandler;

        (document.getElementsByTagName("head")[0] || document.documentElement).appendChild(script_tag);
    } else {
        jQuery = window.jQuery;

3) Widget Controller - (Web Team - Swansea University)

The widget controller receives the course id, calls the getListsByCourseId method in the data access object, assigns the result to a variable used by Mustache to create a html template and finally returns the template using the json content type.

NB: You don't have to use Mustache - you could use any html templating system or just build the html yourself in the controller.


require_once realpath(__DIR__ . '/../../..') . '/config/config.php';

require PATH_TO_HELPERS . '/handle-error.php';

// Validate request

$clean = new stdClass();

if (! array_key_exists('course', $_GET)) {
handleError('HTTP/1.1 400 Bad Request', 'Required parameter "course" not specified');
} else {
$clean->course = (strpos($_GET['course'], "_") !== false)
? preg_replace('/^.*_/', "", $_GET['course']): $_GET['course'];

if (array_key_exists('callback', $_GET)) {
$clean->callback = $_GET['callback'];

// Fetch Data

try {
require_once 'SU/DataAccess/Rebus.php';
$dao = new SU_DataAccess_Rebus();
$view = (object) array(
'lists' => $dao->getListsByCourseId($clean->course, true, true)
} catch (SU_Exception $e) {
handleError('HTTP/1.0 404 Not Found', $e->getMessage());

// Render
$template = file_get_contents(PATH_TO_TEMPLATES . '/rebus/course-reading-list.mustache');
$m = new Mustache();

$data = (object) array(
'html' => $m->render($template, $view)

if (isset($clean->callback)) {
header('Content-Type: text/javascript');
header('X-XSS-Protection: 0');
printf('%s(%s)', $clean->callback, json_encode($data));
} else {

// Used for debugging only
print $data->html;

Example Output:

jQuery15205464451520296695_1354114033050({"html":"<div id=\"rebus-reading-list\"> ..."})

4) Rebus Data Access Object (Web Team - Swansea University)

The Rebus data access object receives a course identifier, builds and calls a url using the Zend HTTP Client, parses the received XML data and returns an array of item data, optionally sorted by the reading list category heading.

Example Output:

Array ( [0] => Array ( [list_id] => 15 [org_unit_id] => 16 [org_unit_name] => Science [year] => 2012 [list_name] => CS-061 Introduction to computing I [published] => y [no_students] => 0 [creation_date] => 1345540250 [last_updated] => 1349271819 [course_identifier] => CS-061 [associated_staff] => [categories] => Array ( [0] => Array ( [title] => Essential reading [items] => Array ( [0] => Array ( [category_heading] => Essential reading [material_type] => Book [title] => Java for everyone / Cay Horstmann. [secondary_title] => [authors] => Horstmann, Cay S., [secondary_authors] => [edition] => [volume] => [issue] => [start_page] => [end_page] => [year] => c2010. [publisher] => John Wiley & Sons, [publication_place] => Hoboken, New Jersey: [publication_date] => c2010. [print_control_no] => 9780471791911 [elec_control_no] => [note] => [creation_date] => 1345540306 [modified_date] => 0 [lms_print_id] => 598756 [lms_elec_id] => [url] => [tags] => ) [1] => Array ( [category_heading] => Essential reading [material_type] => Book [title] => Computer science : an overview / J. Glenn Brookshear. [secondary_title] => [authors] => Brookshear, J. Glenn. [secondary_authors] => [edition] => 10th ed. [volume] => [issue] => [start_page] => [end_page] => [year] => c2009. [publisher] => Pearson Addison-Wesley, [publication_place] => Boston, Mass. ; [publication_date] => c2009. [print_control_no] => 9780321544285 [elec_control_no] => [note] => [creation_date] => 1345540363 [modified_date] => 0 [lms_print_id] => 561530 [lms_elec_id] => [url] => [tags] => ) ) ) ) ) )

5) Data API (PFTS Europe - Rebus List)

Rebus List comes with a very useful API which will return a list of items in a reading list in an xml format when supplied with a valid course code. It can be access using the syntax:

The Widget Output

Default Styling:

Blackboard Styling:


Wednesday, 17 October 2012

Library Camp 2012 - Part 2

Lunch and Cake

When the third session had finished, it was time for lunch which was actually just a continuation of the eating which had taken place throughout the morning sessions - I had been constantly nibbling on the splendid variety of cake and baked produce which accompanies every library camp. Unfortunately, I didn't get my first cup of tea of the day until after 9am and I had therefore begun to revert to my default state during early mornings, that of a Zombie. In hovering around the tea & coffee (yuch) zone however, I was able to spot familiar faces and get first dabs on any new cakes which arrived on the scene.

Though I was trying to curb my consumption of cake in an effort to avoid last year's sugar induced palpitations and out-of-body transcendental experience, I did manage to sample most of the delights on offer. I must mention @rachelsbickley's Rocky Road & @Sonja_Kujansuu's Sweet Potato Pie [0] in dispatches as they were absolutely delicious and provided a much needed breakfast.

Cake! (One of 6 tables)
My contribution to Cake Camp was a Chocolate Guinness Cake - a concept originally brought to my attention via @KelleherBex - for which I received some very kind remarks. When I came to collect the tin at the end of the day, the cake had been reduced to a few crumbs so I'm officially filing the recipe under "success".

Lunch itself was very tasty - in addition to the main offering of rice and chili-con-carni, cold meats, tuna, salad, couscous, bagels, bread rolls and more were on offer.

Sessions 4 & 5

The great thing about a library camp is that do whatever you think you'll get the most out of. I didn't think any of the options on offer during sessions 4 & 5 were particularly relevant to me so I spent the time preparing for the session on Widgets and Web Services which I had proposed on the spur of the moment. It also gave me the chance to have a chat with @benelwell who I knew was working on an analytics project, a topic which I hoped to broach in my session and develop for work.

I was interested to discover that while The Amazing @Daveyp (His official title) at Huddersfield was doing a lot of work analysing database circulation data, Ben was focusing his efforts on analysing data returned by custom JavaScript code [1] from user interaction with Summon. Ben had a nifty set of graphs generated by "R", an open source programming language for statistical computing and graphics [2], by which he was able to demonstrate user behaviour such as the link between the length of the search term and the use of facets.

Fun in the Sun
We were joined throughout the afternoon by various library camp attendees who were taking breaks from the afternoon session for a spot of tea or to enjoy the autumn sun. It was fascinating to watch @joeyanne and @SaintEvelin knitting up a storm whilst @SarahNicolas, @evarol and @sarahgb discussed the virtues of the Sony NEX-5N, a rather spiffing camera which I am seriously considering purchasing.

Session 6: Widgets, Web Services and Libraries - Oh My!

The idea for my session sprung from recent developments at work and the discussion of Open Source Software in the morning session.

At Swansea University, the Web Team have been beavering away creating all and sundry web services for cross-campus consumption with the intention of maximizing the exposure of our most important services. As most of these web services are now complete, we faced a quandary on how best to deliver them. The usual step would be to provide our fellow developers with web service connection details but our resident scrumming, agile, waterfall soaked guru, @g_mawr, suggested that we should consider the use of web widgets. 

From our perspective, a widget is a self-contained, platform agnostic web element. The principle function of a widget is to deliver all the content, style and functionality in one easy to access bundle. Using an article by @amarandon as a basis [3], we are able to deliver a widget with little more than:

<script type="text/javascript" src=""></script>
<div class="widgety-goodness></div> 

I plan to write a complete technical post on @g_mawr's work at a later date.

Putting my library cap on, I think widgets have great potential for library services as they could provide an opportunity for libraries to reach a wider audience through distribution of services and targeted implementation. It could be something as simple as a search box which will direct users to library resources or the addition of library account data to high use sites like Facebook. The embedding of library branding into widgets will also act free advertisement on every site the widget appears on.

Having pitched this idea, the group settled into a discussion of the practicalities of adopting such a practice. I was particularly interested in how the use of widgets might benefit special projects within libraries such as the Health in Mind service represented by @helenkielt. [4]  A brief look at their website immediately suggested two opportunities a) A widget which search the Libraries NI library and eBook Catalogue for relevant mental health material b)The delivery of Health in Mind branded searches on library and local government pages (Health in Mind has a vivid, strong brand which would be perfect for advertising).

Whilst most commentators agreed that the theory and benefits were sound, many, particularly from public library backgrounds lamented the lack of resources and system accessibility which would be required to adopt the widget approach. I had always envisioned library developers working on their own widgets, but several people approached me afterwards inquiring as to the feasibility of creating generic widgets which could be consumed by any library service. As always, this led to a discussion on standards and the pipe dream of a universal Library Management System API which could be used to construct such widgets.

Librarians Chatting (via @SarahNicholas)
The second strand of discussion in the group centred around the use of social media as I suggested the ultimate goal of our project would be to create a Facebook widget, largely because I feel that if we had to choose just one website to get the biggest audience for our users, Facebook would be it. Again, the difference between the public and academic user base was discussed. Though most public libraries are being encouraged to engage with social media sites, only 15% of their users regularly use sites like Facebook or Twitter. With stretched resources, limited funds and staff time, some felt the effort being ask of them was disproportionate to the means available. With that said, most of the libraries represented in the session had a Facebook or Twitter account. The value attached to the accounts varied greatly - some believed it actually the first point of contact with certain user groups whilst others suggested that only library staff engaged with them on a regular basis. A significant proportion lamented the bureaucracy associated with posting to the account with every word and potential meaning scrutinised to such a degree that potential effectiveness was rendered impotent. 

The final part of the session was devoted to the use of library data which has traditionally never been cultivated - circulation data. Every year, libraries across the UK submit it  to various official bodies so that the royalties afforded to authors can be calculated. Whilst this data is lifeblood to firms like Amazon, most libraries are yet to take advantage of it.

Part of the issue may be confusion of ownership - the data often sits in a proprietary database after all and nobody likes playing with their proprietary LMS in case of system failure, table corruption or a hefty service charge. It is therefore up to librarians to demand access to this data in its entirety, not just its raw, possibly undecipherable format but via a consistent standard which will allow use to analyse correlations, check for patterns and construct "Users who loaned this, also loaned..." type services. The emergence of a(n) (inter-)national standard would greatly assist such a project - Amazon may be unwilling to share their schemas or algorithms but it wouldn't hurt to ask!


Thought some of the "otherness" of a Library Camp has now worn off on me, I still managed to get a lot of Library Camp 2012. It is a great opportunity to socialise with a fantastic bunch of people who share a similar occupation but come from all walks of life. The relaxed nature of the event and the personal contact it affords makes learning about and contributing to developments in the sector so much easier.

As a result of Library Camp, I have two new blog posts to write - one on the technical aspects of our deployment of widgets and another on why I think libraries quest for identity is growing stale. I don't profess to have any particularly elucidating remarks on the latter topic and I'm not going to attempt a well-researched scholarly sermon-on-the-mount type discourse - it will be purely my own observations from my limited sphere of experience.

Inspired by @benelwell (and a session by the University of Huddersfield at Gregynog) I also have a renewed enthusiasm for looking into the use of circulation data analytics. I will augment our resource discovery platforms with useful data!

Post Library Camp

The post library camp celebrations continued until 3.45am - Thankfully, I haven't found any photographic evidence of them after 10.30pm.

[1] Based upon the work of @daveyP and @mreidsma

Tuesday, 16 October 2012

Library Camp 2012 - Part 1

On Friday 12th October, I made the journey from Swansea to Birmingham for my third Library Camp at The Signing Tree Conference Centre in Birmingham [0] on Saturday 13 October. For the uninitiated, a Library Camp is an "unconference" style meeting which is free and attendee led with participants pitching ideas for sessions before the event begins.

The theory is that "the sum of the knowledge, experience and expertise of the people in the room is likely to be greater than that of those on the stage at traditional conferences." [1]

The event is accompanied by copious amounts of participant baked goodies and often sandwiched between several opportunities for socialising en-mass. Most attendees get to know about a Library Camp through Twitter so "Twitter Bingo" is a very popular past time between sessions. The game can be particularly difficult if avatars do not match physical realities - at my first event I spent two hours talking to a silver teapot in vain.

Session 1: What is a Library for?

Session Proposals (via @evarol)
In all three Library Camps have attended there has been a session related to the identity of libraries in modern Britain. I dutifully attend because there is often interesting debate and in this instance, there was  smidgen of controversy as the session was facilitated by Ben Taylor of  Red Quadrant, a consultancy company which has been used to "transform" (outsource) [7] public libraries. Once the somewhat hostile crowd's desire to offer up Ben as a sacrificial offering had abated, some interesting comments were made on libraries as an essential element of a democratic society, the role they play in breaking down barriers and prejudices and how we can convince the general public of their value. One commentator was even looking to open a private Tool Library with attached Pizzeria!

Sessions like these often encourage a degree of catharsis among participants who use the opportunity to vent their frustrations (with the added bonus of an "evil corporate punchbag" in attendance). Considering the situation most public librarians find themselves in, this is an understandable and perhaps quite necessary process. For me however, the session was very much one of deja vu. I have been in libraries for ten years and I have heard the same message preached at every library event I have attended. I have therefore resolved not to attend any in the future because I think I have squeezed every last pip of value I am going to get out of them. They will however continue to be essential forums for first time library camper.

Before I say farewell to I would like to make two observations. Firstly, we often talk about convincing the public that libraries have "value". Personally, I do not think this is possible. One can only decide if something has value to oneself from personal experience. Value is an  inherently subjective thing. Libraries would be better off if they decided what they do best,used modern marketing techniques to bring their services to the public attention and allowed individuals to decide if libraries had any value. As libraries have so much to offer, we should have nothing to be afraid of. Secondly, though I recognise that the future of libraries in the current climate is a political matter and think that our library advocates and campaigners are doing a wonderful job, I am uncomfortable with the potential politicisation of libraries. I believe that such a process could alienate many library borrowers and turn them into a party political pawn. Libraries don't belong to any particular aspect of the political spectrum, they transcend it. If certain sections of society or local councillors believe libraries have become irrelevant for modern Britain, it is because people who represented libraries became complaisant with their position within the establishment. (I'll try and put more of what I mean by that statement in a later blog post - hopefully before I get lynched)

Session 2 : Open Source Software

Like the first session, I often attend Open Source Software forums because most of the software I am responsible for is Open Source in nature. The session was chaired by @preater and @liz_jolly and in addition to defining Open Source Software and exploring the differences between OSS and proprietary models of procurement we discussed the cultural changes or cultural shift needed to develop and sustain the use of OSS in libraries, a typically risk-averse environment.

Proposed Sessions
It seems that having "Open Source Advocates" in positions of authority will be key to the process of culture change as the main catalyst for change within existing institutions is the presence of such an individual at a managerial level. I suggested that librarians who had qualified more recently might be more likely to be advocates for OSS, not because of age, but because OSS is trend which library courses might wish to reflect in their syllabus.

Of further interest was the proposal that some kind of framework was required to allow OSS and proprietary systems and support mechanisms to be compared so as to inform the tendering processes which accompany most contract changes. UK Core Specifications already exist for comparing OSS Library Management Systems with proprietary systems - the next logical step would be to produce an open specification which also compared support mechanisms.

Something which I hadn't previously considered was how the use of OSS might effect the culture of a workplace. If responsibility for OSS development remains "in house", a much greater emphasis is placed on local developers - time and money is invested in them rather than a proprietary company. The benefits and pitfalls of this approach are relative but it does have the potential to create a more community based focused to a service. We also  considered how such an approach might effect front-line staff who both consume and support these services.

More interesting points on this discussion have been made by Tattle Tape and Tea, particularly that "barriers to change... often came from the IT department". [2] IT professionals who use OSS enthusiastically at home seem less inclined to implement it in a work environment, largely because of the perceived lack of accountability if a service fails to operate. This doesn't necessarily mean that IT departments will need to take more risks to employ OSS. Perhaps it is more a case of having greater faith in their own abilities, particularly when it comes to designing and implementing recovery procedures and services.

It was interesting to hear about some libraries in Europe which were using Open Source Operating Systems on their public access PCs. Many libraries in the UK have an iMac public network but I have yet to hear of a Unix one despite the fact that many local government and university services (particularly internet sites) will be run by *nix servers.

@preater's recollection of proceedings can be seen on his blog which I heartily recommend as it includes useful definitions from the Free Software Foundation and a more in depth description of the current cultural perception regarding OSS. [3]

Public Engagement in Research and Special Collections

I attended this session because of my recent experience of collaborating with the National Library of Ireland & the National Library of Finland in designing an archives / collection module for VuFind [4] so that our archives could get some of their data to display in a publicly searchable catalogue.

Powering the World Exhibit [5]
I offered a brief synopsis on the project and described how our end product hoped to fuse traditional archival user interfaces with the power of modern search engines and faceting and heard how various archives and libraries are trying to publicise their special collections. Of particular interest was @scapner's Powering the World: Looking at Welsh Industry through Archives project which aims to catalogue and improve access to the outstanding uncatalogued business collections held in Welsh archive repositories. [5] An essential part of the project is a travelling exhibition which is being used to highlight its aims and showcase some of its materials. Archives and Special Collections may find it useful to set up similar touring exhibitions or simply exchange advertising with local libraries as a way of engaging a wider audience with their material.

The discussion then progressed to the differences between academic and public collections during which The Hive, a partnership between The University of Worcester and Worcester County Council, was mentioned. [6] In a nutshell, The Hive is a joint academic and public library which also houses archives and special collections. If it is successful in overcoming the inherent differences in academic and public provision (particularly with regards to licensing of resources like journals, ebooks etc and the potential need for distinct academic / public areas), I think may similar partnerships are likely to follow.

It is interesting to note that the catalogue search page for The Hive has three distinct options from three different systems - one for books, one for archives and one for archaeology reports - each with their own different interfaces. The integrated approach our VuFind module takes will hopefully eradicate our need for this whilst maintaining the option for a different "skin" for an archive only search if desired.

That's the end of part one of my Library Camp 2012 review. Part two will feature cake, analytics, widgets and web services.

[7] See the comment from Ben below - I think my mistake highlights the problem of relying on hearsay and not checking the facts oneself! 

Sunday, 29 January 2012


Insomnia but not Faithless
I have of late been suffering from a prolonged bout of insomnia. The term is relative to me as in general, I'm a very poor sleeper. It usually takes me 2 - 3 hours to get to sleep but since November last year, if I do get to sleep, it's taking 3 - 6 hours.

The problem is that I just can't turn my brain off. It is constantly brining up some memory, imagining some scenario or attempting to solve some problem which presents itself in 'self-talk'. The only time I can get to sleep with 5 or 10 minutes if hitting the sack is if I've had a few drinks but alcohol induced sleep is of poor quality and ultimately detrimental to one's health.

When I do manage to get to sleep, I often have quite disturbing dreams featuring all sorts of monsters so even that is not at all restorative. One of my latest nocturnal horrors featured transdimensional arachnid beings who were kidnapping people and ingesting them via a nauseating process of liquefaction.

I have tried every trick and recommendation in the book to try and get some decent kip but all to no avail. A while back, I thought the gym and Ovaltine were working wonders but now even that mighty combination has been overwhelmed.

I am therefore extremely lucky that my job involves the use of flexi time which means my extreme morning lethargy doesn't cost me.

I actually remember the last time I had a good night's sleep where I woke incredibly refreshed. It was during the summer of 2009 - I woke up at 7.30 to a cheery chorus of birdsong led by an enchanting blackbird and a soft breeze which was just catching the curtains. Now that is the stuff that dreams are made of!