Wednesday, 5 March 2014

Adventures in Zend Framework 2: Episode 3: Major Features

ZF2 Structure

A quick glance at the ZF2 directory structure instantly reveals its primary focus as a framework designed with web applications in mind. In addition to regular requirements for software development like caching, authentication, logging, cryptography and MVC components, you will so find classes for forms, http connections, soap and XmlRpc.

Zend Framework 2 directory structure
In this post, I'm going to give an overview of two of ZF2's most important components, the Event Manager and the Service Manager, an understanding of which is key to expanding the usefulness of the framework to your application.

The Event Manager

Perhaps the biggest change from ZF1 to ZF2 was the focus on event driven architecture. The whole application flow depends on events - a change in the application triggers an event and listeners can be attached to each event so that further changes can be made when desired. Each listener is usually passed information about the event, thus allowing it to set logic as to how to proceed given different event variables. Thus for example, one could attach listeners to the dispatch.error event which could log information about the error or send an e-mail to a system administrator.

Most of the events listeners attach to will be associated with the MVC flow but it is possible to create custom events and listeners which can be triggered manually and listened to accordingly. Custom events require their own event manager and are triggered with a name plus optional parameters for a target object, an array of arguments and a callback function.

use Zend\EventManager\EventManager

$event = new EventManager('star-wars');
$event->trigger('death star destruction', null, array('rebels' => array("Luke Skywalker", "Han Solo", "Princess Leia"));

Event listening is generally established during the Bootstrap MVC phrase, often in the OnBootstrap method of Module.php. As module boostrapping can require a lot of code, VuFind 2 adds a custom Bootstrapper class which allows it to create different methods for each initiation requirement such as forcing the user to an error page if the system is unavailable. A listener is added to an event via the attach method which accepts the event name (or an array of event names), an optional callback and a priority as arguments.

class Module {
    public function onBootstrap(MvcEvent $e)
        $bootstrapper = new Bootstrapper($e);



class Boostrapper {

public function __construct(MvcEvent $event)
        $this->event = $event;
        $this->events = event->getApplication()->getEventManager();

    public function bootstrap()
        // automatically call all methods starting with "init":
        $methods = get_class_methods($this);
        foreach ($methods as $method) {
            if (substr($method, 0, 4) == 'init') {


    protected function initSystemStatus()
        // If the system is unavailable, forward to a different place:
        if (isset($this->config->System->available)
            && !$this->config->System->available
        ) {
            $callback = function ($e) {
                $routeMatch = new RouteMatch(
                    array('controller' => 'Error', 'action' => 'Unavailable'), 1
            $this->events->attach('route', $callback);

MVC events have their own event manager which is why in the code above, we can instantly call $this->events->attach(). If you have created your own events however, you need to access them via the shared event manager. Using the VuFind code as a base, we could therefore do something like:

    protected function initHolonet()

        $callback = function ($e) {
            $rebels = $e->getParam('rebels');
            $sm = $e->getApplication()->getServiceManager();
            $holonet = $sm->get('holonet');

            return $holonet->transmit('', $rebels);
        $sharedManager = $this->events->getSharedManager();
'death star destruction', $callback);

The priority argument is extremely useful as it allows us to stack our listeners according to the order in which they should run. Calling stopPropagation(true) on an event will also stop any subsequent listeners from executing, something which is very handy when an error occurs or a dependency is not available. The result of the listener callback is returned as an argument to the callback of the event trigger which allows for further processing if required.

The Service Manager

If the Event Manager determines when you run your code, the Service Manager is primarily designed to help with how you run it. The Service Manager is essentially a registry of services which can be intelligently initiated with a simple key. In the initHolonet example above, the Service Manager was accessed via
$sm = $e->getApplication()->getServiceManager();
and the holonet service instance was loaded with


Though it is possible to define a service almost anywhere, it makes sense to do so either in the module configuration file or the module class itself for ease of reference.

The service manager accepts the following primary configuration keys:
1) invokeables
Classes which can be invokes without configuration
2) factories
Objects which require configuration, often with other service manager services
3) abstract Factories
Factories that can can create multiple services based on the name supplied
4) services
Objects which have already been instantiated
5) initializers
Services which initialise other services whenever they are created
6) aliases
Alias a key to a known service key
7) shared
Determines whether or not a service should be shared or if a new service should be created with every call
8) allow_override
Whether or not the services can be overridden by other modules
When defined in module.config.php, a service manager configuration might look something like this:
return array(

    service_manager => array(

    'invokables' => array(
        'Anakin' => '\Empire\Personel\AnakinSkywalker';

        'Empire\Transmitter' => \Empire\Resources\Transmitter

    'factories' => array(
        'holonet' => function ($sm) {
            $transmitter =
            return new \Empire\Services\Holonet($transmitter);

    'abstract_factories' => array(

    'services' => array(
        'light' => new \Force\LightSide(),

        'dark' => new \Force\DarkSide()

    'initializers' => array(
        function ($instance, $sm) {
            if ($instance instanceof


    'aliases' => array(
        'DarthVader' => 'Anakin';

    'shared' => array(
        'holonet' => false,


View Helpers, Controllers, Controller Plugins and Form Elements effectively have their own service managers and their configurations needs to be returned with their own keys instead of "service_manager". These keys are "view_helpers", "controllers", "controller_plugins" and "form_elements".

If you wished to achieve something similar in the Module Class for a service manager configuration, you can use the getServiceConfig method to return a configuration array:

public function getServiceConfig() {
   return array(
     'services' => array(
       'light' => new \Force\LightSide(),
       'dark' => new \Force\DarkSide()


The getViewHelperConfig(), getControllerConfig(), getControllerPluginConfig() and getFormElementConfig() methods are also available for their service managers.

Service Manager: Basic Functionality

When  a call is made to the get method of the service manager with a service key, the service manager checks its registry of instances, invokables, factories, aliases and abstract factories for that key. If the key is found, it calls its create method with that key which in turn instantiates the service according to the service type (invokable, factory etc).


if (isset($this->instances[$cName])) {
    return $this->instances[$cName];


if (!$instance) {
    if (
        || isset($this->factories[$cName])
        || isset($this->aliases[$cName])
        || $this->canCreateFromAbstractFactory($cName, $name)
    ) {
        $instance = $this->create(array($cName, $name));
    } elseif ($usePeeringServiceManagers && !$this->retrieveFromPeeringManagerFirst) {
        $instance = $this->retrieveFromPeeringManager($name);


Once instantiated, the service manager also applies the initializers to each instance, either through the initializer initialize method or via the supplied function.


foreach ($this->initializers as $initializer) {
  if ($initializer instanceof InitializerInterface) {
     $initializer->initialize($instance, $this);
  } else {
     call_user_func($initializer, $instance, $this);

The initalize method might therefore look something like:

public function initialize($instance, ServiceLocatorInterface $serviceLocator) {

    if ($instance instanceof \Force\ForceAwareInterface) {

    if (method_exists($instance, 'setHolonet')) {


Service Manager: Abstract Factories

Abstract Factories effectively allow you to create your own logic for instantiating service classes. They must implement Zend\ServiceManager\AbstractFactoryInterface which requires that you define a canCreateServiceWithName method and a createServiceWithName method. VuFind defines it's own AbstractPluginFactory as follows:

abstract class AbstractPluginFactory implements AbstractFactoryInterface
    protected $defaultNamespace;

    protected $classSuffix = '';

    protected function getClassName($name, $requestedName)
        // If we have a FQCN, return it as-is; otherwise, prepend the default prefix:
        if (strpos($requestedName, '\\') !== false
            && class_exists($requestedName)
        ) {
            return $requestedName;
        // First try the raw service name, then try a normalized version:
        $finalName = $this->defaultNamespace . '\\' . $requestedName
            . $this->classSuffix;
        if (!class_exists($finalName)) {
            $finalName = $this->defaultNamespace . '\\' . ucwords(strtolower($name))
                . $this->classSuffix;
        return $finalName;

    public function canCreateServiceWithName(ServiceLocatorInterface $serviceLocator,
        $name, $requestedName
    ) {
        $className = $this->getClassName($name, $requestedName);
        return class_exists($className);

    public function createServiceWithName(ServiceLocatorInterface $serviceLocator,
        $name, $requestedName
    ) {
        $class = $this->getClassName($name, $requestedName);
        return new $class();

The key here is that the getClassName method is used to determine the class name, largely via a combination of default namespaces and class suffixes. This makes it easy to create a number of services which share either a common namespace, common class suffix or combination of both. It's also possible to pass in and instantiate a fully qualified namespace.

If desired, it would also be possible to initialise class dependencies as part of the createServiceWithName method. As the Service Locator is passed in as the first argument, it would be easy to access other services. Zend applications will be easier to debug and maintain if you stick to the same principles for all your services. If you decide to initialise dependencies as part of the createServiceWithName for one abstract factory, always initialise dependencies for your abstract factories there. For my money, the best place to perform your initialisation is with the specifically designed initialiser option of the Service Manager. You can either have one initializer responsible for all your instances or you can create multiple initializers according to instance type. If these are all referenced in module.config.php, they will be easy to find and understand.

Service Manager: Plugin Manager

The Plugin Manager is an extension of the service manager which allows users to establish a collection of services according to a specific set of criteria, perhaps via interfaces or ancestor class. It automatically registers an initializer which should be used to verify that a plugin instance is of a valid type, provides for a validatePlugin method and accepts an array of options for the constructor, which can be used to configure the plugin when retrieved.

VuFind extends the Zend Plugin manager to include a getExpectedInterface, thus forcing anything that extends it to implement that method. The validePlugin method then checks to make sure that the plugin is an instance of the value returned by the getExpectedInterface method to ensure plugin integrity.

public function validatePlugin($plugin)
    $expectedInterface = $this->getExpectedInterface();
    if (!($plugin instanceof $expectedInterface)) {
        throw new ServiceManagerRuntimeException(
            'Plugin ' . get_class($plugin) . ' does not belong to '
            . $expectedInterface

abstract protected function getExpectedInterface();

Once instantiated, a user can then call get on the plugin manager to retrieve the desired instance.


Coming Up...

My next post will look at some of the basics of ZF2's MVC implementation.

Tuesday, 4 March 2014

Adventures in Zend Framework 2: Episode 2: The Basics

You've Been Framed!

Zend Framework 2 (ZF2) describes itself as "the most popular framework for modern, high-perfoming PHP applications" [1]. It is an open source project which is primarily used for developing web applications and services which uses "100% object-oriented code and utilises most of the new features of PHP 5.3, namely namespaces, late static binding, lambda functions and closures." [2] It is essentially a library of components which can be used separately or together, designed to work most effectively within an MVC implementation.

My first experiences of ZF2 came through utilising VuFind 2 [3] and much of what I am going to write about ZF2 is done so through the prism of VuFind. With that said, I have begun to use ZF2 in other projects and some of the experience I have of certain components such as Forms and Access Control Lists has come through my own experimentation.

Funny Bones

The first port of call for anyone interested in ZF2 (after reading as much literature as possible) should probably be the Skeleton Application available at It offers a step-by-step guide to getting a ZF2 application started using the most common components like databases and forms and gives a basic introduction to the MVC concept. As part of the installation process, the user will also be encouraged to use Composer [4], a dependency manager for PHP which makes adding and using third party software extremely easy. I also recommend familarising yourself with Git [5] and Github [6] as the preferred version control system.


The Model-View-Controller software pattern to which ZF2 adheres is basically a means of arranging code into logical but integrated sections, separating a developer's representation of information "from the ways that information is presented to or accepted from the user". [7] The model encapsulates the core code complete with the logic and functions required to generate or manipulate data, the view constitutes any outputted representation of this information whilst the controller effectively acts as a  go-between the model and the view, accepting and converting user input into commands for each.

Explaining how the MVC process works in ZF2 could take up the entire contents of a book, let alone a blog post. At the beginning, I suffered some sleepless nights as my brain tried to make sense of some terribly complex diagrams which I managed to find online. Anything I offer here will necessarily be a gross simplification but it may at least offer a base for further exploration.

1) Bootstrap
All the requirements for running the application (including dependencies) are prepared
2) Route
A user request is matched to a controller and action
3) Dispatch
The controller processes the application logic chain
4) Render
The information is represented to the user
5) Finish
The MVC process is complete
Each of these "phases" have different "hooks" which allow a developer to attach or inject logic when required. Some "hooks" are already predefined (such as just before and just after rendering takes place) but a developer can also add their own. In ZF2, these "hooks" are actually "events" (as it made clear by naming conventions which use "pre", "post", "on" etc as descriptions) and an "Event Listener" is responsible for dealing with the separate MVC phases. I will cover the Event Manager and listening to events in a later post.

This MVC structure is partly revealed in the directory structure of the Skeleton Application:

A simplified flow of  information through a ZF2 application would therefore run something like this:
  1. A user makes a request via a url
  2. Web server redirects force all requests via the index.php file in the public folder (unless the requested filename actually exists e.g. images, css files etc)
  3. index.php begins the auto loading process by requiring init_autoloader.php and initialising any dependencies in the vendor directory (including ZF2 which is a dependency of the Skeleton Application)
  4. index.php begins the MVC process, using the configuration settings found in config/application.config.php. Settings include an array of module namespaces (e.g. "Application"), the locations in which modules should be found (e.g. "modules" and "vendor"), overriding configuration files and options for caching.
  5. Modules are loaded and initialised via the Module Manager, largely through the operation of Module.php (which contains "onBootstrap" and "init" methods plus methods for establishing the module configuration and library location e.g. module/Application/src/Application) and config/module.config.php. Module configuration files are merged with the application configuration to create the final configuration.
  6. The Service Locator (responsible for locating predefined services) plus the Router, View and Event services are set up so that they can be used in the Bootstrap phase.
  7. After bootstrapping, the MVC logic is processed as the route is determined from the url, a controller and action are selected, the model code is executed, the response is determined (and rendered if required) and the request is finished.
Coming Up...

In my next post, I'll highlight some of ZF2's major features. It was only once I'd got my head around them that the ZF2 penny began to drop.


Monday, 3 March 2014

Adventures in Zend Framework 2: Episode 1: An Ode to VuFind


When I first began working for Swansea University in 2009, I have to admit, I was more of a scripter than a developer. Having studied history at university and arriving in the job with no official qualifications in computing, my suitability for the role was based entirely on personal study and experimentation.

My knowledge of front end web desgin and PHP had largely been acquired  via a general computing interest, a year's stint in a primary school I.C.T. department, several years experience in Swansea Public Library Service and the odd request to build a website for a friend or acquaintance.

Fortunately, the primary piece of software for which I was to be responsible - VuFind [1] - was written in PHP. By examining the source code and using the excellent user community, I was able to demonstrate enough familiarity with the VuFind to propose a development path to suit the goals of the project.

VuFind: OOPs I did it again

The first lesson I received from VuFind (or more accurately from the user community, particularly Demian Katz, the project leader) was Object Orientated Programming (OOP) [2]. Up until that point, I had only ever used PHP's native functions, perhaps augmented with a few libraries which had been recommended in "Teach Yourself" type books which were accessed via includes.

Moving from scripting to OOP was probably the most important step I have taken in the last five years. OOP not only makes developing applications far more efficient as frequently used code can be put into methods to be utilised whenever required, it also opens up a world of tried and tested methodologies which can speed up the development process, resulting in code which is more robust, portable and easier to debug.

For the first time, I wasn't just writing code to get things done - I was writing code mindful of considerations such as accepted standards, performance, portability, repository integration and usefulness to others. It felt good to be able to make some small contributions to the direction of the VuFind project and to see some of my patches integrated into the core repository.

VuFind: The Gift that keeps on giving

VuFind 1.0 had introduced me to a world of inheritance, dependency injection, modular programming, encapsulation, interfaces and abstracts. It had also forced me to learn more about the Apache web server, to take my first steps into Unix via a Virtual Box installation of Ubuntu and to consider repository management via SVN. By the time VuFind was launched for the South West Wales Higher Education Partnership (SWWHEP), I was responsible for building and configuring an Ubuntu server from scratch on a VMWare platform.

My experience on the SWWHEP project was also responsible for landing me a permanent job at Swansea University as part of the Web Team. Again, this was a fantastic opportunity as I would be working with professional developers who were willing to help me fill in the gaps which still existed as a result of my rather ad hoc career. If VuFind primarily helped expand what I was able to develop, the move to the web team greatly improved how I was able to develop code. I began to use an integrated development environment for the first time, switched to Unix via Fedora and was introduced to concepts such as test driven development, automated deployment and continuous integration.

VuFind & Zend

VuFind 1.0 had a loose, custom MVC framework. As the project began to grow however, the user community decided that it would be beneficial to base future developments on a tried and tested framework. VuFind 2.0 was initially developed on Zend Framework 1.8 but as Zend Framework 2 was released during this process, VuFind was eventually re-factored to take advantage of these developments. I had never used a framework before and when the beta versions of VuFind 2.0 were released, I realised that Zend 2 used many concepts outside of my comfort zone. As with OOP however, thanks to Demian Katz and the VuFind community, I was able to ask questions and get very helpful answers. Working with the web team also meant that I could tap into their knowledge to resolve particular issues.

Using the experience gained working with VuFind 2, I plan to write a series of brief articles on using the Zend 2 platform. I don't expect to provide any great revelations or insights as I am definitely a consumer rather than an innovator when it comes to programming. Nevertheless, the articles should at least serve me as "revision notes" and they may save some poor soul from hours agonising over an issue which has a simple solution, something which became apparent to me only after the fact!


Friday, 19 July 2013

I'm an alien I'm a legal alien, I'm a Shambrarian at an Archives Conference

I don't drink wine I take real ale my dear,
I like my data nice and flat,
You can see it in my resource discovery tools,
I'm a Shambrian at an Archives Conference

A Pilgrim in an unholy land?

Between Thursday 27th June and Friday 28th of June, I was fortunate to attend and present at the Apex Conference 2013: Building infrastructures for archives in a digital world at Trinity College, Dublin. The opportunity arose out of the collaborative project I was involved with together with the National Library of Ireland (@NLIreland) and the National Library of Finland  add archival data to VuFind.

Archivists discover two librarians shambrarians in their midst?
There's a scene in Indiana Jones and the Last Crusade where Indiana and his father end up in Berlin to reclaim the latter's Grail diary. Upon arriving at a Nazi rally, Henry Jones Senior exclaims "My boy, we are pilgrims in an unholy land". I have to admit to a little trepidation at gate crashing an archives conference, suffering as I do with virtual archive illiteracy. The first session I attended by Daniel Pitti on "The emerging archival metadata landscape" almost had me reaching for my emergency parachute as it was a veritable orgy of acronym dropping. Had I been playing the buzz word bingo drinking game I would have been paralytic by 10.40.

Thankfully, my fears were soon dispelled as I realised that the concepts of data use I learned from libraries are universally applicable and that the folk gathered in Dublin were a thoroughly decent bunch, despite their use of strange acronyms and nested data sets. I should also clarify that the Apex Conference was nothing like a Nazi rally - far from it - no books or precious manuscripts were burned in the process.

"What do you think of Elves now, Sam?" ... "I reckon there’s Elves and Elves. They’re all elvish enough, but they’re not all the same."

How I imagined an archivist conference
So says Samwise Gamgee to Frodo Baggin in Lord of the Rings and the same is largely true of anyone working with cultural data. As I listened to each of the presentations, it struck me that Librarians, Archivists and [collective term for Museum curators] all have the same problems in classifying items and their solutions are largely the same - the major differences necessitated by the form of the items they are describing. Libraries largely deal with individual items which do not require context whilst archives deal with items where context is critical. Context and meaning is achieved by a cataloguing system which represents this nature, making it accessible to interested parties and archives do this according to geographical location, author (in the loosest sense of the word) and item. For an outsider looking in, it is easy to be overwhelmed especially as there appear to be many competing standards [1], but this is the nature of the archival beast. The challenge that Apex (@APEx_project) has taken on it to take this complexity and create standards and best practice which will allow archives across Europe to present their collections in a meaningful way to a digital world which is not necessarily the purveyance of experts. For archives and archivists, this is essential as accessibility is becoming a critical measure of value. For interested parties, as full paid up members of Generation Google, it is de rigueur.

Fascinating Jim...

I attempt to scan some archivists

Taken as a whole, I thoroughly enjoyed my foray into the world of archives. Archives have enough in common with libraries to ensure that I wasn't completely at a loss and exploring their differences kept things interesting. The technical nature of some of the presentations likewise perked my interest and there are clearly some very talented people working with archival data. In total, I attended 16 presentations on day one and 4 sessions on day two. It's not possible to do each presentation justice in a blog post so I'll limit my review to some of my favourites.

Maud Medves's presented on the CENDARI Project ( @CendariProject) which aims to integrate digital archives for medieval and modern European history. In developing their product, they needed to produce a custom EAG Cendari data structure which was producing using an innovative use of Jira & SVN.

It was great to get some perspective on Apex from Susanne Waidmann who gave a great exposition of its funding, structure and staffing through to its purpose, tools and value and this was followed by Bastiaan Verhoef's review of Apex's primary output, the Archives Portal Europe, was particularly intriguing as it revealed that the back end of the software was very much the same as VuFind, as it utilised the powerful Solr search server.  Apex make all of their EAD tools available for free and even if one is not submitting data to the Archives Portal, they may be of use for local archives who wish to normalise their data.

I was astounded by the amount of work one man development team Jochen Graf had put into the Monasterium Collaborative Archive which collects historical documents as digital copies and provides an interface to both view an annotate the digital copies in an open, collaborative framework. This was the first in many presentations which highlighted projects which are using crowd sourcing to augment and enrich data. Similarly, @petralinks suggested on Twitter, "Facilitating multilingual descriptions is a repeating theme in all presentations on metadata and standards for digital archives".

The first session in the afternoon was given by Julia Fallon of the marvellous Europeana project (@EuropeanaEU) which collects digital objects from Europe's leading galleries, libraries, archives and museums,  making them available under agreed licences. The presentation was primarily about open data and licensing but I was mostly intrigued by the way in which Europeana handled digital objects. Europeana really has embraced social networking with aplomb and has an active presence on Twitter, Facebook, Pinterest and Google Plus. The talented folks who work there have also developed a media rich, free iPad App which is well work a download from the App Store.

As if Europeana wasn't enough to get me drooling, Julia then went on to mention,one of Europeana's mini-projects which records and preserves 89 first-hand accounts from individuals who experienced the fall of communism and the reunification of Europe in Poland, Latvia, Germany, Estonia, Hungary and the Czech Republic. As part of the project, they submitted their data to History Pin which uses a map rich interface to allow users to explore history at a particular time period by geographical location.

European 1989: History Pin

After a brief hiatus where an angry archivist shouted a presenter from The Historical Archives of the Pontifical Gregorian University for being denied access last year, the theme of captivating projects was continued by Constanza Gianccini's expose on which aims to "map and publish in a critical edition the extensive correspondence of European intellectuals with the Swiss cultural historian Jacob Burckhardt over a period of more than half a century, from 1842 to 1897". Burckhardt Source uses to allow users to add metadata to its letters. appears to be an incredibly powerful open source tool which effectively creates a collaborative research community which may not be limited to subject specialists. By giving web documents semantic context, they can be enriched with data from countless sources.

Given the clear successes that most of these projects were experiencing through integrating social media into their platforms, it was apt that the first session of Friday morning was presented by Laura Gould (@LauraLGould) of the Lothian Health Services Archive (@lhsaeul) and Guinevere Barlow of The Carmichael Watson Project (@coll97CW) who talked about the use of social media in their workplace. Both use Facebook, Twitter and Flikr to augment their website and the key to the increased interest they have received in their projects is interaction; having a social media presence is not enough - users want to engage with experts once their curiosity has been perked by a particular post or article. A benefit of this approach is that it has actually attracted staff to internships at Lothian Health Services Archive who regard working with social media as a desirable employment opportunity.

The last session I attended at the Apex Conference was given by Tom Cobbaert () on ArchiefWiki, a community for Dutch archivists (which happens to have a Wiki) centred around opening up data and sharing knowledge. ArchiefWiki supports the Archivist’s 2.0 Manifesto which essentially makes it a kind of Fight Club for Archivists.

Tom Cobbaert at an ArchiefWiki event

Finally, on Friday Afternoon, my experience of universe as inhabited by archivists sadly came to an end. After a lengthy chat with my fellow presenter Eoghan O'Carragain of The National Library of Ireland (@NLIreland)* on digital repositories over a cup of Barry's Tea, it was time to board my plane back to Cardiff where I had the following thoughts:

1) Aer Arran really need to work on how they tell passengers that a part on the plane is "defective" (The engine, the pilot?)
2) Archivists and Librarians have a lot in common
3) The Apex conference was full of fun and talented people
4) Individual projects really are achieving great things but greater collaboration and openness could greatly reduce the amount of money and effort institutions have to put into their projects
5) I wasn't able to find one archivist willing to have a photo taken with me in white gloves

An archivist in traditional attire
* another institution which makes fabulous use of Twitter

Wednesday, 28 November 2012

Widgets, Web Services and Libraries - Oh My!

The time has come to fulfill my promise [0] to write a blog post on our use of widgets at Swansea University.


To cut a long story short, the decision to focus on developing widgets came about through the realisation that the Web Team were being asked increasingly to provide data to a number of disparate university web based systems such as Blackboard, iFind Discover (Our library catalogue powered by VuFind and Voyager) iFind Research (Our remote discovery service powered by Xerxes and Metalib / SFX) and our intranet and web pages. We therefore wanted to create a service which followed a recognised pattern, required very little technical proficiency to implement on the client side and was easy to share and distribute. Thankfully, our lead developer Geraint always has his finger on the pulse so he was able to suggest that widgets, self-contained, platform agnostic web elements, would meet all our needs, allowing us to deliver content, style and functionality in one easy to access bundle.

The fundamental basis of approach was taken form an article by Alex Marandon [1]. In our environment, this equates to something like:

1) Data API (e.g. Rebus List - our reading list solution from PTFS Europe)
2) Data Access Object (Web Team Developed object which queries the API)
3) Widget Controller (Loads the Data Access Object, processes data, returns data in required format)
4) Widget JavaScript (Collects variables, sends them to the controller via ajax and uses a callback function to process the results)
5) Client added script tag and target html element:

<script type="text/javascript" src=""></script>
<div class="widgety-goodness" data-id="1234"></div>

This method of delivering content places virtually all of the development requirements for a service on the web team. All a client needs to be able to do is embed two lines of code in their web page with any required variables. The only further developments a client may wish to undertake is to create custom css styles or JavaScript / jQuery behaviours.

A Guided Example

For those more interested in the technical details of the service, I will now try to outline the development of our Rebus List Widget with a particular emphasis on modifications we have made to Alex Marandon's original post.


Get a particular course reading list from Rebus List by supplying a valid course code


1) Script tag and HTML element (Blackboard Team - Swansea University)

The Blackboard team embed the required script and html elements to each course page and generate the course id. They have also created their own style sheet which will override the default css supplied by the widget.


<script type="text/javascript" src=""></script>
<div class="rebus-course-reading-list-widget" data-course="CS-061">&nbsp;</div>

2) Widget JavaScript (Web Team - Swansea University)

The Widget JavaScript grabs the course id from the html element attribute and submits it via Ajax to the Widget controller. If successful, it loads the returned content in to the html element in addition to grabbing required resources like html5shiv [3] or css files.

The major differences in our JavaScript files compared to those suggested is that we have formatted them using JSLint [4] and added methods to deal with problems caused by the way IE7 loads css and js files.

This particular example does not have any jQuery actions associated with the delivered content but it could be added as part of the $.getJSON method if required.


/*jslint browser: true, devel: true, sloppy: true */

(function () {
    var jQuery,
        main = function () {

            jQuery(function ($) {

                var $targets = $('.rebus-course-reading-list-widget'),
                    devEnv ='env=dev') !== -1,
                    baseUrl = devEnv ? '/widgets' : '//',
                    styleSheet = baseUrl + '/css/rebus/course-reading-list.css';

                    ? document.createStyleSheet(styleSheet)
                    : $('head').append($('<link>', {
                        rel:  'stylesheet',
                        type: 'text/css',
                        href: styleSheet
                // Do we need html5shiv?
                if ($.browser.msie && $.browser.version < 8) {
                    script_tag = document.createElement('script');
                    script_tag.setAttribute("type", "text/javascript");
                    script_tag.setAttribute("src",  baseUrl + "widgets/vendor/html5shiv/html5shiv.js");
                    (document.getElementsByTagName("head")[0] || document.documentElement).appendChild(script_tag);

                $targets.each(function () {
                    var $this = $(this),
                        course = $this.attr('data-course'),
                        uri = baseUrl + '/controllers/rebus/course-reading-list.php?callback=?&course=' + course;

                    $.getJSON(uri, function (data) {

        scriptLoadHandler = function () {
            jQuery = window.jQuery.noConflict(true);

    if (window.jQuery === undefined || window.jQuery.fn.jquery !== '1.5.2') {
        script_tag = document.createElement('script');
        script_tag.setAttribute("type", "text/javascript");
        script_tag.setAttribute("src", "//");

        if (script_tag.readyState) { // old-IE
            script_tag.onreadystatechange = function () {
                if (this.readyState === 'complete' || this.readyState === 'loaded') {
        } else {
            script_tag.onload = scriptLoadHandler;

        (document.getElementsByTagName("head")[0] || document.documentElement).appendChild(script_tag);
    } else {
        jQuery = window.jQuery;

3) Widget Controller - (Web Team - Swansea University)

The widget controller receives the course id, calls the getListsByCourseId method in the data access object, assigns the result to a variable used by Mustache to create a html template and finally returns the template using the json content type.

NB: You don't have to use Mustache - you could use any html templating system or just build the html yourself in the controller.


require_once realpath(__DIR__ . '/../../..') . '/config/config.php';

require PATH_TO_HELPERS . '/handle-error.php';

// Validate request

$clean = new stdClass();

if (! array_key_exists('course', $_GET)) {
handleError('HTTP/1.1 400 Bad Request', 'Required parameter "course" not specified');
} else {
$clean->course = (strpos($_GET['course'], "_") !== false)
? preg_replace('/^.*_/', "", $_GET['course']): $_GET['course'];

if (array_key_exists('callback', $_GET)) {
$clean->callback = $_GET['callback'];

// Fetch Data

try {
require_once 'SU/DataAccess/Rebus.php';
$dao = new SU_DataAccess_Rebus();
$view = (object) array(
'lists' => $dao->getListsByCourseId($clean->course, true, true)
} catch (SU_Exception $e) {
handleError('HTTP/1.0 404 Not Found', $e->getMessage());

// Render
$template = file_get_contents(PATH_TO_TEMPLATES . '/rebus/course-reading-list.mustache');
$m = new Mustache();

$data = (object) array(
'html' => $m->render($template, $view)

if (isset($clean->callback)) {
header('Content-Type: text/javascript');
header('X-XSS-Protection: 0');
printf('%s(%s)', $clean->callback, json_encode($data));
} else {

// Used for debugging only
print $data->html;

Example Output:

jQuery15205464451520296695_1354114033050({"html":"<div id=\"rebus-reading-list\"> ..."})

4) Rebus Data Access Object (Web Team - Swansea University)

The Rebus data access object receives a course identifier, builds and calls a url using the Zend HTTP Client, parses the received XML data and returns an array of item data, optionally sorted by the reading list category heading.

Example Output:

Array ( [0] => Array ( [list_id] => 15 [org_unit_id] => 16 [org_unit_name] => Science [year] => 2012 [list_name] => CS-061 Introduction to computing I [published] => y [no_students] => 0 [creation_date] => 1345540250 [last_updated] => 1349271819 [course_identifier] => CS-061 [associated_staff] => [categories] => Array ( [0] => Array ( [title] => Essential reading [items] => Array ( [0] => Array ( [category_heading] => Essential reading [material_type] => Book [title] => Java for everyone / Cay Horstmann. [secondary_title] => [authors] => Horstmann, Cay S., [secondary_authors] => [edition] => [volume] => [issue] => [start_page] => [end_page] => [year] => c2010. [publisher] => John Wiley & Sons, [publication_place] => Hoboken, New Jersey: [publication_date] => c2010. [print_control_no] => 9780471791911 [elec_control_no] => [note] => [creation_date] => 1345540306 [modified_date] => 0 [lms_print_id] => 598756 [lms_elec_id] => [url] => [tags] => ) [1] => Array ( [category_heading] => Essential reading [material_type] => Book [title] => Computer science : an overview / J. Glenn Brookshear. [secondary_title] => [authors] => Brookshear, J. Glenn. [secondary_authors] => [edition] => 10th ed. [volume] => [issue] => [start_page] => [end_page] => [year] => c2009. [publisher] => Pearson Addison-Wesley, [publication_place] => Boston, Mass. ; [publication_date] => c2009. [print_control_no] => 9780321544285 [elec_control_no] => [note] => [creation_date] => 1345540363 [modified_date] => 0 [lms_print_id] => 561530 [lms_elec_id] => [url] => [tags] => ) ) ) ) ) )

5) Data API (PFTS Europe - Rebus List)

Rebus List comes with a very useful API which will return a list of items in a reading list in an xml format when supplied with a valid course code. It can be access using the syntax:

The Widget Output

Default Styling:

Blackboard Styling:


Wednesday, 17 October 2012

Library Camp 2012 - Part 2

Lunch and Cake

When the third session had finished, it was time for lunch which was actually just a continuation of the eating which had taken place throughout the morning sessions - I had been constantly nibbling on the splendid variety of cake and baked produce which accompanies every library camp. Unfortunately, I didn't get my first cup of tea of the day until after 9am and I had therefore begun to revert to my default state during early mornings, that of a Zombie. In hovering around the tea & coffee (yuch) zone however, I was able to spot familiar faces and get first dabs on any new cakes which arrived on the scene.

Though I was trying to curb my consumption of cake in an effort to avoid last year's sugar induced palpitations and out-of-body transcendental experience, I did manage to sample most of the delights on offer. I must mention @rachelsbickley's Rocky Road & @Sonja_Kujansuu's Sweet Potato Pie [0] in dispatches as they were absolutely delicious and provided a much needed breakfast.

Cake! (One of 6 tables)
My contribution to Cake Camp was a Chocolate Guinness Cake - a concept originally brought to my attention via @KelleherBex - for which I received some very kind remarks. When I came to collect the tin at the end of the day, the cake had been reduced to a few crumbs so I'm officially filing the recipe under "success".

Lunch itself was very tasty - in addition to the main offering of rice and chili-con-carni, cold meats, tuna, salad, couscous, bagels, bread rolls and more were on offer.

Sessions 4 & 5

The great thing about a library camp is that do whatever you think you'll get the most out of. I didn't think any of the options on offer during sessions 4 & 5 were particularly relevant to me so I spent the time preparing for the session on Widgets and Web Services which I had proposed on the spur of the moment. It also gave me the chance to have a chat with @benelwell who I knew was working on an analytics project, a topic which I hoped to broach in my session and develop for work.

I was interested to discover that while The Amazing @Daveyp (His official title) at Huddersfield was doing a lot of work analysing database circulation data, Ben was focusing his efforts on analysing data returned by custom JavaScript code [1] from user interaction with Summon. Ben had a nifty set of graphs generated by "R", an open source programming language for statistical computing and graphics [2], by which he was able to demonstrate user behaviour such as the link between the length of the search term and the use of facets.

Fun in the Sun
We were joined throughout the afternoon by various library camp attendees who were taking breaks from the afternoon session for a spot of tea or to enjoy the autumn sun. It was fascinating to watch @joeyanne and @SaintEvelin knitting up a storm whilst @SarahNicolas, @evarol and @sarahgb discussed the virtues of the Sony NEX-5N, a rather spiffing camera which I am seriously considering purchasing.

Session 6: Widgets, Web Services and Libraries - Oh My!

The idea for my session sprung from recent developments at work and the discussion of Open Source Software in the morning session.

At Swansea University, the Web Team have been beavering away creating all and sundry web services for cross-campus consumption with the intention of maximizing the exposure of our most important services. As most of these web services are now complete, we faced a quandary on how best to deliver them. The usual step would be to provide our fellow developers with web service connection details but our resident scrumming, agile, waterfall soaked guru, @g_mawr, suggested that we should consider the use of web widgets. 

From our perspective, a widget is a self-contained, platform agnostic web element. The principle function of a widget is to deliver all the content, style and functionality in one easy to access bundle. Using an article by @amarandon as a basis [3], we are able to deliver a widget with little more than:

<script type="text/javascript" src=""></script>
<div class="widgety-goodness></div> 

I plan to write a complete technical post on @g_mawr's work at a later date.

Putting my library cap on, I think widgets have great potential for library services as they could provide an opportunity for libraries to reach a wider audience through distribution of services and targeted implementation. It could be something as simple as a search box which will direct users to library resources or the addition of library account data to high use sites like Facebook. The embedding of library branding into widgets will also act free advertisement on every site the widget appears on.

Having pitched this idea, the group settled into a discussion of the practicalities of adopting such a practice. I was particularly interested in how the use of widgets might benefit special projects within libraries such as the Health in Mind service represented by @helenkielt. [4]  A brief look at their website immediately suggested two opportunities a) A widget which search the Libraries NI library and eBook Catalogue for relevant mental health material b)The delivery of Health in Mind branded searches on library and local government pages (Health in Mind has a vivid, strong brand which would be perfect for advertising).

Whilst most commentators agreed that the theory and benefits were sound, many, particularly from public library backgrounds lamented the lack of resources and system accessibility which would be required to adopt the widget approach. I had always envisioned library developers working on their own widgets, but several people approached me afterwards inquiring as to the feasibility of creating generic widgets which could be consumed by any library service. As always, this led to a discussion on standards and the pipe dream of a universal Library Management System API which could be used to construct such widgets.

Librarians Chatting (via @SarahNicholas)
The second strand of discussion in the group centred around the use of social media as I suggested the ultimate goal of our project would be to create a Facebook widget, largely because I feel that if we had to choose just one website to get the biggest audience for our users, Facebook would be it. Again, the difference between the public and academic user base was discussed. Though most public libraries are being encouraged to engage with social media sites, only 15% of their users regularly use sites like Facebook or Twitter. With stretched resources, limited funds and staff time, some felt the effort being ask of them was disproportionate to the means available. With that said, most of the libraries represented in the session had a Facebook or Twitter account. The value attached to the accounts varied greatly - some believed it actually the first point of contact with certain user groups whilst others suggested that only library staff engaged with them on a regular basis. A significant proportion lamented the bureaucracy associated with posting to the account with every word and potential meaning scrutinised to such a degree that potential effectiveness was rendered impotent. 

The final part of the session was devoted to the use of library data which has traditionally never been cultivated - circulation data. Every year, libraries across the UK submit it  to various official bodies so that the royalties afforded to authors can be calculated. Whilst this data is lifeblood to firms like Amazon, most libraries are yet to take advantage of it.

Part of the issue may be confusion of ownership - the data often sits in a proprietary database after all and nobody likes playing with their proprietary LMS in case of system failure, table corruption or a hefty service charge. It is therefore up to librarians to demand access to this data in its entirety, not just its raw, possibly undecipherable format but via a consistent standard which will allow use to analyse correlations, check for patterns and construct "Users who loaned this, also loaned..." type services. The emergence of a(n) (inter-)national standard would greatly assist such a project - Amazon may be unwilling to share their schemas or algorithms but it wouldn't hurt to ask!


Thought some of the "otherness" of a Library Camp has now worn off on me, I still managed to get a lot of Library Camp 2012. It is a great opportunity to socialise with a fantastic bunch of people who share a similar occupation but come from all walks of life. The relaxed nature of the event and the personal contact it affords makes learning about and contributing to developments in the sector so much easier.

As a result of Library Camp, I have two new blog posts to write - one on the technical aspects of our deployment of widgets and another on why I think libraries quest for identity is growing stale. I don't profess to have any particularly elucidating remarks on the latter topic and I'm not going to attempt a well-researched scholarly sermon-on-the-mount type discourse - it will be purely my own observations from my limited sphere of experience.

Inspired by @benelwell (and a session by the University of Huddersfield at Gregynog) I also have a renewed enthusiasm for looking into the use of circulation data analytics. I will augment our resource discovery platforms with useful data!

Post Library Camp

The post library camp celebrations continued until 3.45am - Thankfully, I haven't found any photographic evidence of them after 10.30pm.

[1] Based upon the work of @daveyP and @mreidsma

Tuesday, 16 October 2012

Library Camp 2012 - Part 1

On Friday 12th October, I made the journey from Swansea to Birmingham for my third Library Camp at The Signing Tree Conference Centre in Birmingham [0] on Saturday 13 October. For the uninitiated, a Library Camp is an "unconference" style meeting which is free and attendee led with participants pitching ideas for sessions before the event begins.

The theory is that "the sum of the knowledge, experience and expertise of the people in the room is likely to be greater than that of those on the stage at traditional conferences." [1]

The event is accompanied by copious amounts of participant baked goodies and often sandwiched between several opportunities for socialising en-mass. Most attendees get to know about a Library Camp through Twitter so "Twitter Bingo" is a very popular past time between sessions. The game can be particularly difficult if avatars do not match physical realities - at my first event I spent two hours talking to a silver teapot in vain.

Session 1: What is a Library for?

Session Proposals (via @evarol)
In all three Library Camps have attended there has been a session related to the identity of libraries in modern Britain. I dutifully attend because there is often interesting debate and in this instance, there was  smidgen of controversy as the session was facilitated by Ben Taylor of  Red Quadrant, a consultancy company which has been used to "transform" (outsource) [7] public libraries. Once the somewhat hostile crowd's desire to offer up Ben as a sacrificial offering had abated, some interesting comments were made on libraries as an essential element of a democratic society, the role they play in breaking down barriers and prejudices and how we can convince the general public of their value. One commentator was even looking to open a private Tool Library with attached Pizzeria!

Sessions like these often encourage a degree of catharsis among participants who use the opportunity to vent their frustrations (with the added bonus of an "evil corporate punchbag" in attendance). Considering the situation most public librarians find themselves in, this is an understandable and perhaps quite necessary process. For me however, the session was very much one of deja vu. I have been in libraries for ten years and I have heard the same message preached at every library event I have attended. I have therefore resolved not to attend any in the future because I think I have squeezed every last pip of value I am going to get out of them. They will however continue to be essential forums for first time library camper.

Before I say farewell to I would like to make two observations. Firstly, we often talk about convincing the public that libraries have "value". Personally, I do not think this is possible. One can only decide if something has value to oneself from personal experience. Value is an  inherently subjective thing. Libraries would be better off if they decided what they do best,used modern marketing techniques to bring their services to the public attention and allowed individuals to decide if libraries had any value. As libraries have so much to offer, we should have nothing to be afraid of. Secondly, though I recognise that the future of libraries in the current climate is a political matter and think that our library advocates and campaigners are doing a wonderful job, I am uncomfortable with the potential politicisation of libraries. I believe that such a process could alienate many library borrowers and turn them into a party political pawn. Libraries don't belong to any particular aspect of the political spectrum, they transcend it. If certain sections of society or local councillors believe libraries have become irrelevant for modern Britain, it is because people who represented libraries became complaisant with their position within the establishment. (I'll try and put more of what I mean by that statement in a later blog post - hopefully before I get lynched)

Session 2 : Open Source Software

Like the first session, I often attend Open Source Software forums because most of the software I am responsible for is Open Source in nature. The session was chaired by @preater and @liz_jolly and in addition to defining Open Source Software and exploring the differences between OSS and proprietary models of procurement we discussed the cultural changes or cultural shift needed to develop and sustain the use of OSS in libraries, a typically risk-averse environment.

Proposed Sessions
It seems that having "Open Source Advocates" in positions of authority will be key to the process of culture change as the main catalyst for change within existing institutions is the presence of such an individual at a managerial level. I suggested that librarians who had qualified more recently might be more likely to be advocates for OSS, not because of age, but because OSS is trend which library courses might wish to reflect in their syllabus.

Of further interest was the proposal that some kind of framework was required to allow OSS and proprietary systems and support mechanisms to be compared so as to inform the tendering processes which accompany most contract changes. UK Core Specifications already exist for comparing OSS Library Management Systems with proprietary systems - the next logical step would be to produce an open specification which also compared support mechanisms.

Something which I hadn't previously considered was how the use of OSS might effect the culture of a workplace. If responsibility for OSS development remains "in house", a much greater emphasis is placed on local developers - time and money is invested in them rather than a proprietary company. The benefits and pitfalls of this approach are relative but it does have the potential to create a more community based focused to a service. We also  considered how such an approach might effect front-line staff who both consume and support these services.

More interesting points on this discussion have been made by Tattle Tape and Tea, particularly that "barriers to change... often came from the IT department". [2] IT professionals who use OSS enthusiastically at home seem less inclined to implement it in a work environment, largely because of the perceived lack of accountability if a service fails to operate. This doesn't necessarily mean that IT departments will need to take more risks to employ OSS. Perhaps it is more a case of having greater faith in their own abilities, particularly when it comes to designing and implementing recovery procedures and services.

It was interesting to hear about some libraries in Europe which were using Open Source Operating Systems on their public access PCs. Many libraries in the UK have an iMac public network but I have yet to hear of a Unix one despite the fact that many local government and university services (particularly internet sites) will be run by *nix servers.

@preater's recollection of proceedings can be seen on his blog which I heartily recommend as it includes useful definitions from the Free Software Foundation and a more in depth description of the current cultural perception regarding OSS. [3]

Public Engagement in Research and Special Collections

I attended this session because of my recent experience of collaborating with the National Library of Ireland & the National Library of Finland in designing an archives / collection module for VuFind [4] so that our archives could get some of their data to display in a publicly searchable catalogue.

Powering the World Exhibit [5]
I offered a brief synopsis on the project and described how our end product hoped to fuse traditional archival user interfaces with the power of modern search engines and faceting and heard how various archives and libraries are trying to publicise their special collections. Of particular interest was @scapner's Powering the World: Looking at Welsh Industry through Archives project which aims to catalogue and improve access to the outstanding uncatalogued business collections held in Welsh archive repositories. [5] An essential part of the project is a travelling exhibition which is being used to highlight its aims and showcase some of its materials. Archives and Special Collections may find it useful to set up similar touring exhibitions or simply exchange advertising with local libraries as a way of engaging a wider audience with their material.

The discussion then progressed to the differences between academic and public collections during which The Hive, a partnership between The University of Worcester and Worcester County Council, was mentioned. [6] In a nutshell, The Hive is a joint academic and public library which also houses archives and special collections. If it is successful in overcoming the inherent differences in academic and public provision (particularly with regards to licensing of resources like journals, ebooks etc and the potential need for distinct academic / public areas), I think may similar partnerships are likely to follow.

It is interesting to note that the catalogue search page for The Hive has three distinct options from three different systems - one for books, one for archives and one for archaeology reports - each with their own different interfaces. The integrated approach our VuFind module takes will hopefully eradicate our need for this whilst maintaining the option for a different "skin" for an archive only search if desired.

That's the end of part one of my Library Camp 2012 review. Part two will feature cake, analytics, widgets and web services.

[7] See the comment from Ben below - I think my mistake highlights the problem of relying on hearsay and not checking the facts oneself!