ajm-old

thoughts from a co-worker in the kingdom of culture

Building the Knowledge & Learning Beta »

Hi I’m Rob Lee, senior technical architect for Knowledge & Learning (K&L) at the BBC.

I’m part of the team building the new Knowledge & Learning product. In this post I’m going to describe the technical work we’ve undertaken so far, some of which you can see in the new Beta release that my colleague Chris Sizemore recently blogged about.

Video content on the Knowledge and Learning Beta Video content on the Knowledge & Learning Beta


The Knowledge & Learning product aims to bring together the variety of factual and learning content we have on the BBC into a coherent proposition. In the formal learning space we currently have a number of sites including Bitesize, Skillswise and Class Clips amongst others.

There are tens of thousands of content items across these sites with each site having different mechanisms for publishing, discovering and describing the content it serves.

Rather than approach the development of the new product as a ‘big-bang’ exercise trying to incorporate everything at once, we’ve adopted an incremental approach to adding content and building out new functionality using short form audio and video clips as the first content type to surface in the Beta.

This meant the team had a well-defined problem to solve but still had to consider some key challenges inherent in building the new product:

  1. Multi-device consumption: Many of the existing sites were built before smartphones and tablets became commonplace so a method of delivering content in a multi-device world was required.
  2. Content management and migration: Existing sites use a collection of content production systems to author content. We needed to rationalise these and define a content migration process.
  3. Information architecture and content modelling: Each of the existing sites have similar but different ways of describing and navigating to their content. We needed a consistent approach to describe content in the new product.


In each of these areas we tried to build on existing successes and key systems at the BBC, allowing the team to focus on delivering new product features.


Technical Overview

image1_1024.jpg Architecture overview


The overview diagram above shows the components involved in the initial Beta release. The BBC has a standard platform for dynamic content rendering that follows a multi-tier approach, keeping a clear separation of concerns.

The rendering layer is built on a standard PHP framework (Zend) with BBC extensions for common capabilities and components. Access to the data layer is via Java service components with rendering components never communicating directly with the data layer.

Caching is a key feature of the stack with reverse proxy caches (Varnish) proxying requests through to the PHP page assembly (rendering) layer using standard HTTP caching mechanisms to allow caching of pages.

The service layer has a similar caching arrangement and both rendering and service layers have access to memcached which provides a distributed memory object caching system to applications.

Ensuring the Java service components are performing is key as these are aggregated in the page assembly layer, meaning the effects of a slow service component can be amplified when a series of requests are made from the page assembly layer.

Memcached is useful in this scenario however, we use caching carefully, where it will have the biggest impact as it adds another layer of indirection making failure modes more difficult to diagnose.

We’ve tried to keep the architecture simple in line with our initial goal for the Beta but with the longer term aims still in mind. I’ll talk about some of the components involved in the diagram in the sections below.


Serving content in a multi-device world

image_3_1024.jpg The Beta site across different devices


One of the emerging patterns in delivering content to the variety of devices people now use is Responsive Design, a way to build a single fluid experience that works across multiple devices rather than a series of separate disconnected designs.

This required a mobile first approach to design and implementation which was a new way of working for the team. One of the benefits that Responsive Design delivers is that when we build a new feature in the product it’s available on release to all devices at the same time rather than having to add a feature for each device class.

We followed the responsive BBC News model of delivering a core experience to HTML4 browsers and an enhanced experience to HTML5 capable browsers that ‘cut the mustard’. We use CSS3 media queries to support responsive styling for different viewports (screen sizes) and device orientations which, for example, allows font size on a large-screen tablet in landscape mode to be increased when compared to that on a smaller mobile device.

This has given us a framework for delivering all new Knowledge & Learning content in a responsive way which we can continue to improve and build on. This is still a work in progress and we’re looking at ways we can improve our initial content footprint and the way we deliver assets.

We are one of the first collection of sites at the BBC to use the new Responsive Barlesque (ostensibly the header and footer used for every BBC page) called ORB. Using ORB made it easy to deliver a single fluid experience as it follows the same responsive principles as our application.

The responsive approach also makes it easier to use standard platform caching technologies to effectively manage increased site hits during busy times, such as exam revision times, as the product can deliver small cacheable HTML pages to clients with appropriate cache control information. This means the load on platform infrastructure can be kept to a minimum.


Content management and migration image_4_640.jpg Diagram showing the content management systems and how they interact with the site


Where appropriate we’ve used standard content production systems within the BBC allowing the team to concentrate on building out new functionality in the product.

For short form audio and video clips we use iBroadcast2 which provides a way to publish short form clips in a standard way so the content is available on as wide a selection of devices as possible when used in combination with the BBC media player (known as emp).

Using this system means we’re part of the same media ecosystem that powers BBC pages like /programmes and BBC iPlayer meaning the product benefits from ongoing improvements.

For general content management we use iSite which is a standard content management system within the BBC and is also used to content manage the BBC Internet Blog, see Welcome to the new look Internet blog for more information.

With these core systems in place we set about planning the migration of the 10,000+ short form audio and video clips from the existing class clips site. There were a number of challenges we had to address in order to migrate the video and associated content:

  • Clip format: The clips were originally transcoded using systems primarily intended for content consumed on the desktop. This meant finding a way to re-encode the clips in the same formats produced by iBroadcast2.
  • Clip content: The existing clips content (e.g. Classroom Ideas) was held in a legacy publishing system and needed to be extracted and placed alongside the clips published in iBroadcast2.
  • Clip mappings: The curriculum information held against the existing clips (e.g. its subject or topic) needed to be mapped in the curriculum data model.


A simplified process flow for the clip migration is shown in the figure below: 

imge_5_640.jpg Simplified clip migration process flow


The migration has three key steps:

  1. Transcode clips and push into the media playout platform.
  2. Generate and validate learning metadata and content for clips.
  3. Transform learning metadata and content inputs and push into the content management system.


The clip transcoding was carried out by the Media Publishing System (MPS) team who ingested the source clips, bulk processed them and added them to the same systems that iBroadcast2 publishes to, with clips transcoded to the same formats. This means in the future all clips (both migrated and newly added) can be editorially managed using iBroadcast2.

The K&L editorial teams then produced a series of controlled vocabularies which were mapped to each clip. The teams also validated the associated content for each clip, amending and rewriting where needed to ensure the content was suitable for the new product.

Finally, a transformation component was produced that took these inputs, generated XML documents, ready for addition to iSite so they could be rendered in the new Beta site.


Describing content

image_6_640.jpg The curriculum ontology


Following on from the work on Dynamic Semantic Publishing we’ve adopted a similar approach, moving away from a relational publishing model to one that separates semantics from content and allows for easier re-use.

The existing learning sites have no single model to describe content that could be reused in the new product. The team therefore produced a model of the UK national curricula that allows description of all learning (and other) content in context. This model will contain over 2000 curriculum topics across the various key stages and levels.

The key classes in the model are:

  • Level: A stage of education e.g. KS3.
    Field of Study: A subject discipline of a curriculum e.g. Science.
  • Topic: A more specific description of a learning resource in the context of a field of study e.g. Energy.
  • Programme of Study: A combination of an educational Level and the Field of Study e.g. KS3 Science.
  • Topic of Study: A Topic in the context of a Programme of Study e.g. KS3 Physics Energy.


As demonstrated in the diagram above, the model allows for modelling of complex relationships where both KS3 Physics and KS3 Geography share a common Topic ‘Energy’.

We’ve released this model for others to use which you can find at the curriculum ontology page. The model is licensed under an open license meaning others can reuse and adapt it for their own purposes.

We’d be interested in any feedback or comments from anyone who adopts this model.

This curriculum data is held in the BBC’s Linked Data Platform (LDP) expressed as Resource Description Framework (RDF) which will allow a learning context to be applied to many content items produced by the BBC in the future.

We’ve created an LDP-powered domain-specific API for the curriculum domain model, that powers each of the curriculum pages (e.g. the KS3 Geography Environment page) and the curriculum navigation on the site.

We’re also aiming to expose this domain model as structured data in the content. In our next release we’ll be adding in-page mark-up from the educational vocabulary in Schema.org which is contributed from the LRMI project.

Schema.org is a “collection of schemas, i.e. HTML tags, that webmasters can use to mark-up their pages in ways recognized by major search providers”. LRMI is an education-specific vocabulary, providing a way to consistently describe educational content.

Practically this means that machines (such as search engines) will be able to index and understand more about the content from the structure, aiding it’s discovery. We’ve created mappings between Schema.org and the curriculum ontology allowing us to expose the data in the curriculum ontology in a way that search engines (and others) can easily consume.

This mapping is still a work in progress and will evolve as Schema.org and LRMI evolve but we believe it’s a useful way to begin exposing the semantics for learning content at the BBC


What’s next?

We’re currently working on adding more clips to the Beta site to give further coverage across the UK curricula. We’re aiming to add localisation support for UK indigenous languages as well as adding new content formats to the site.

If you have any thoughts or queries that you’d like to feedback then please leave a comment below or email us at knowledgeandlearningfeedback@bbc.co.uk.

Robert Lee is senior technical architect for BBC Knowledge & Learning.

http://bit.ly/139IQjw

Designing Experiences for Responsive Web Sites »

Before You Get Started

Responsive Web design is intended to ensure that a site’s layout and content scale fluidly to the available screen real estate. This is a great approach for focusing your investments on improving site content and user functionality while ensuring that users have a good experience regardless of what device and screen size they use to visit your site. If you didn’t read the first article in this series, “Why the Web Is Ready for Responsive Web Design,” be sure to read it first.

It’s worth taking a step back, however, to think through your site’s experience and understand whether the device with which a user accesses your site changes the user’s expectations of the site’s functionality. Is the user checking your site for quick updates with her cellphone while she’s on the go? Is he sitting down, 10 feet away from a large TV screen, looking to immerse himself in a relatively passive consumption experience of rich content, videos and games? Are other users sitting down at their PCs, looking to get the most from your site content? Most of all, how do these expectations affect the site layout and functionality that you provide at those corresponding screen sizes?

What Kind of Site Is This?

Planning the content hierarchy for your site across different form factors is definitely the first step to having a great responsive-site experience. Consider the following examples, which evaluate and compare the top experiences that customers want to have when they access your site from a 4-inch phone while they walk or take public transportation, when they’re sitting at their computer desk, and when they’re lounging on their couches in their living rooms.

News Site (Content Consumption)

People visit ContosoNews.com primarily to do one thing—catch up on the day’s current affairs. When you see how this site is presented on a PC screen, it’s designed to have a layout like a newspaper. More important, the single home page is expected to attract and retain different kinds of readers, with interests in current affairs, business, sports, entertainment and other topics, and show them that ContosoNews has content that will interest them. The home page has a rich layout with slide shows, cycling of recommended articles, various categories of news available below the fold if you scroll down, recommended editorials and even the weather. Figure 1 shows a schematic illustration of the site at different resolutions.
Comparing Layouts for ContosoNews.com

Figure 1. Comparing Layouts for ContosoNews.com

If you visit this site on your mobile phone browser, you see a subset of the content, with menu and link navigation to the remaining content. The content that was available on the PC has been prioritized, and the top headline has been given focus above the fold. The slide show of recommended articles is replaced by a series of blurbs with links. The top articles from the Other Categories section are gone, replaced with a single category picker that navigates away from the home page.

In this way, users visiting the site on a phone can, in a cursory glance, become aware of the content available for consumption and dig deeper at their convenience.

Local Attraction (Hyper-Local Site)

Contoso Station is a hip new restaurant in Seattle. When people visit the restaurant’s site on their PC or TV screen, the restaurant proudly shows its latest Yelp reviews, news articles and tweets from users who add the hashtag #i&lt3contoso.

However, when you visit the site on a smartphone, the company makes a fair assumption that you’re visiting its site on the go with hopes to find its location, hours of operation and phone number. The phone might even request your location and show you a map with the quickest route to the restaurant. Some of the remaining content can be presented with much less detail—for example, the Yelp reviews are boiled down to one-line snippets—and the rest of the content (the Twitter feed, for example) can be hidden altogether for users visiting the site on their phones. Figure 2 shows an example of this scenario.

Comparing Layouts for Contoso Station
Figure 2. Comparing Layouts for Contoso Station

As seen in Figure 2, local businesses should prioritize and show an entirely different set of content to phone users and make their mobile experiences more sensitive to location.

Media Site (Rich Audiovisual Content)

ContosoTube is a popular Internet service where people share all kinds of videos. Users can see the latest top-rated and most frequently watched content. As they sign-in and explore the site, they can create and edit playlists of videos, get personalized recommendations, subscribe to other users’ playlists and even send each other messages.

The experience of ContosoTube on a phone is geared toward showing videos that a user has opened from other apps (instant messages, email, Twitter and so on), searching to view a video, and letting logged-in users access their existing subscriptions and playlists. Their experience is very limited for content curation.

What’s interesting about ContosoTube is that the Xbox site experience is similar to the phone experience from a user-functionality perspective, although the Xbox site is laid out differently based on screen real estate because even when ContosoTube users visit the site on their large screens, they are probably accessing it from their living room and doing so with controls less precise than a mouse. While the screen size of the TV might tempt developers to provide a more PC-like experience in terms of available functionality, it would be highly likely that users accessing ContosoTube on their TVs would focus primarily on watching content and not on creating it, managing it and messaging with others. Figure 3 compares site layouts for ContosoTube.

ContosoTube on a PC, TV and Smartphone.
Figure 3. ContosoTube on a PC, TV and Smartphone.

HTML5 Games

On Build New Games, a website that explores HTML technologies for creating immersive gaming experiences on the browser, Jack Lawson provides a great discussion about what a gaming experience might be like for a responsively designed Web site.

A game is a great example of site design where users expect entirely different experiences based on the context in which they visit the site. For example, if a user visits the site WorldOfContosoCraft.com from his PC, he probably expects a full-fledged gaming experience—he can play the game himself, interact and communicate socially with other players through the in-game chat feature, make customizations and settings to his avatar and even participate in the in-game marketplace to buy upgrades, armor and other goodies.

On the console, this user might expect a similarly feature-rich experience, but he would also have expectations about being able to use his controller to drive the experience instead of the mouse and keyboard. (There are currently libraries for Chrome and Firefox—although with limited cross-browser support, as Nikhil Suresh points out in his discussion of controller support in JavaScript libraries.)

On the phone itself, the user might be looking to perform simpler actions, such as checking up on his inventory and gamer stats, performing some customizations on the avatar and maybe buying some add-ons from the in-game marketplace. Game developers, who can provide such a contextually relevant experience to users who visit their site from their cellphone for a few minutes, can keep their users engaged in the overall experience even when they can’t play the game.

Considerations for UI Design (aka Fat Fingers)

In addition to information design, you need to think about modes of user input. Today, first and foremost, this means that your site UI be touch-friendly. Visitors are not using touch for your Web site only on phones and tablets; they also use touch-screen-based PCs. Moreover, when you think about users on the Xbox, they’re interacting with the UI elements of your Web page by using a joystick, which is not as precise as a mouse.

Ideally, you do not want to design and code your user interface elements (buttons, links, form controls and so on) differently for touch (tablets and phones) than for PCs with traditional mouse-keyboard elements. In fact, Windows 8 makes this distinction nonexistent, with users able to run Microsoft Surface with a USB mouse as well as desktops with touch-screens. Moving forward, it’s reasonable to assume that more traditional PCs will be equipped with touch-screen functionality.

That’s why the best approach is to design a one-size-fits-all interface for user inputs that is comfortable for touch users to access. Mouse and keyboard users can still interact with these pages just fine.

To highlight some paradigm shifts in this approach, let’s take the example of one of the most common forms of navigation, the drop-down menu, on my favorite local radio station, Contoso Music. (See Figure 4.) This is just one example of a solution to links and navigation menus for touch, but it illustrates the most important considerations we need to take.

The Drop-Down Navigation Menu for Contoso Music

Figure 4. The Drop-Down Navigation Menu for Contoso Music

This navigation menu has a couple of issues that go beyond responsive layout, but they are still an integral part of building a unified site experience that scales across multiple devices.

  1. First, a lot of sites use navigation menus on which links are revealed when a user mouses over the menu titles. This is absolutely unacceptable because a mouse-over does not translate well to touch browsers. In fact, touch-input aside, you shouldn’t rely on a mouse-over to reveal any useful information at all because it is not keyboard accessible and goes against W3C accessibility guidelines.
  2. Second, look at the relative sizes of the links Playlists and DJs. These two pieces of information are supposed to be at the same level in the hierarchy. However, the size of the link is determined by the size of the text. This makes the DJs link less prominent, and also harder to precisely tap on a touch-screen. The DJs link could be as small as 20 px by 40 px, which is not accessible.
  3. Another subtle problem, which you can see by glancing at the menu list items, is that only the text items themselves are hyperlinks. Here again, the touch user would be better served if the target for the link Foo was the entire width of the flyout menu instead of just the text width.

Moreover, users on all-in-one devices might utilize the same machine in different device configurations, in which case they might access your site with a mouse at one point and then revisit it later by using touch. It’s beneficial to provide the user with touch-friendly, well-spaced hyperlinks and navigation.

A common example of touch-friendly navigation that lots of sites use for their menus, especially on mobile apps or in a sidebar for tablets, is shown in Figure 5.

A Touch-Friendly Redesign of the Contoso Music Navigation Menu

Figure 5. A Touch-Friendly Redesign of the Contoso Music Navigation Menu

The navigation menu utilizes touch, mouse or keyboard to expand and collapse the accordion-style submenus. All the links are the same width (even the submenu items), and for each link, the entire rectangle is clickable, not just the text.

A good example of a site that has made this transformation is MSN.com. The old MSN.com (shown in Figure 6) sports a significantly higher content density, with lots of text links (with smaller clickable areas) that are tightly packed (creating room for error when using touch and gaming joysticks), as well as a mouse-over to reveal the subcategories of news (see the menu under Entertainment).

The Old MSN.com

Figure 6. The Old MSN.com

Figure 7 shows the new touch-friendly version of MSN.com. While currently offered only on Windows 8, the touch-friendly UI will be rolled out across the board for all browsers after testing. Notice the more spacious layout and larger hit targets.

The New Look for MSN.com

Figure 7. The New Look for MSN.com

One Site Fits All

Responsive Web design should not only be about resizing the same content gracefully based on user screen sizes. To best connect with your users across multiple screens, your site should not only be aware of the device’s physical characteristics (such as screen size) but also infer the user’s physical circumstances, modes of input and the kind of information she is seeking.

In the next article in this series, I’ll cover some implementation techniques for responsive design.

This article is part of the HTML5 tech series from the Internet Explorer team. Try-out the concepts in this article with three months of free BrowserStack cross-browser testing @ http://modern.IE.

http://bit.ly/Yz9Fjn