Silverback has been our goto tool when recording user testing sessions. Silverback 2 had some issues with updated MacBooks and couldn’t get access to the built in webcam, so we would have to use an external webcam. Silverback 3 fixed this issue along with bringing a host of other new features. Unfortunately we encountered some issues with Silverback 3 and it has now being pulled from the app store. Hopefully Silverback 3 will be back soon and be as reliable and robust as previous versions. In the meantime it seems the perfect opportunity to explore some alternatives to Silverback.
Before we begin exploring alternatives it would be useful to understand what functionality we require and what criteria are of importance. We are looking for a tool that captures screen recording and audio, combined with the output from the built in webcam. It needs to be compatible with the latest MacBooks, the built in microphone and webcam. We will be reviewing the tools based on the following key criteria:
Price - The cost needs to be affordable (£100s is ok £1000s not so much)
Reliability - Reliability is essential - We need confidence that recordings are being created as expected
Simplicity - Has to be easy to use, particularly for people who won’t be performing testing every day
The first option we are going to review is called Lookback. Lookback has various different versions you can download including:
For our requirements we are purely interested in the OS X application and associated web platform.
Installation of the OS X application was a straight forward process. Once installed I started by creating a test recording and exploring the interface. There were a few bits of unexpected behaviour, by default you have the ability to restart the previous recording, this is actually a really handy feature. Visibility of mouse clicks is off, so if you want these it’s important to remember to enable them.
I then previewed the recording locally and it all seemed to work as expected providing a combined screen capture and webcam recording. My next step was to save or export the recording. To do this you have to first upload the recording. I’m always a little wary about using third party platforms to store user recordings, but the process itself was straightforward.
After uploading the video to the web app I found it in “my recordings”. Opening the video launched a youtube like player. At first it seemed like the video didn’t load, but skipping the timeline ahead a couple of seconds seemed to fix this.
A couple of interesting features of the web app include the ability to create organisation and within this create playlists. So you could create a playlist for each project.
You can also add time based comments on the video via a simple interface. Then when you playback the video the comments pop up at the appropriate point. This would be really useful when reviewing a session with the client or wider team.
Sharing seems to be a focus of the web platform. You can invite people to collaborate on your videos and have the ability to easily embed a video on a website. Providing functionality to easily discuss the recordings with collaborators.
My next step was to export and download a combined video that included:
Opening the video in VLC media player worked fine. Trying to open with quickTime player encountered some issues as the file didn’t have a file extension. Manually adding the file extension .mov allowed me to open the file in quickTime player.
Lookback seems to provide the basic features that we need. I’m slightly concerned about having to upload videos before being able to export, both due to the complexities and data privacy. With this being a Beta release it will be interesting to see what price range the final release falls into. Regarding reliability it’s really too early to tell, but first impressions are promising.
After using the app to capture real user testing sessions I encountered problems accessing my sessions. When you complete a recording the session attempts to upload automatically. Unfortunately my uploads got stuck, which stopped me from exporting the final combined recordings. This seemed to be primarily down to the size of the recordings. For the capture of a high resolution MacBook screen, web cam and audio, a 45 minute session was clocking in at over a 7gb. A single day of testing could leave you with over 35gb of recordings attempting to upload. For our purposes I don’t think it will be practical to upload so much data. Until there is an offline option I don’t think this solution is fit for what we require.
This issue highlights a recurring problem with all screen capture solutions, which is the amount of space recordings take, when capturing high resolution screens. Keeping a careful eye on your disk space is essential, particularly if you are capturing the sessions to the drive that contains your operating system (your machine may start to grind to a halt). If you run out of disc space, then transferring data and making room can be a time consuming process.
Price: 5/5 - Free (Beta) during Beta. No final price available.
Reliability: 2/5 - Major issues with uploading of the videos. Apart from this issue the app seemed pretty stable and robust, with only a few crashes on launching.
Simplicity: 4/5 - Interface is simple and intuitive. Basic tasks are straight forward. I made a couple of mistakes the first time I used the app, but you quickly learn how it works.
When conducting and analysing qualitative user research it is essential to be able to explore the research topic and identify relevant findings. Often we are in an environment where we are working to a tight time span, and don’t have the luxury to review research sessions numerous times, watching and analysing video recordings. In these scenarios I have found it essential to code and theme my research. This helps to identify trends in research and also makes your notes much easier to interpret, which is particularly useful when facilitating the sessions on your own. This article will explore these coding schemes and provide insight into how you might generate appropriate themes.
When referring to qualitative user research, I am primarily referring to user testing - one to one user testing sessions following a semi structured interview methodology.
The coding scheme used evolves over time and is tailored to each individual project, dependant on th goal of the research. Following a basic structure for the majority of research, coding may include the following: identifying and numbering tasks, classifying observed behaviours, participant quotes, non verbal language, experience issues etc. Example notation/output of a coding schema:
 - Struggling to find filters, takes time to find result [O] [I] [3:45]
This is made up of the following:
 - Number of the task the user was trying to complete (as per the interview script)
Struggling to find filters… - Description of user behaviour
[O] - Indicates this was an observed behaviour
[I] - Indicates this is an issue that is impacting task completion time/rate. [3:45] - The estimated time stamp of the behaviour
By applying this simple code to your research notes, it is now much easier to review and categorise your findings, rewatching the recording where appropriate.
Identifying themes throughout the research process is essential to the research achieving its goals. A theme can be described as an idea you want to validate or a repeating idea that emerges throughout the research process. Themes should help answer pertinent questions relating to the goal of the research. They can be formed before, during or after the research sessions. Typically you will start with an understanding of what you want to find out, from this you can identify some high level themes. Throughout the research process you would iterate, modify and discover further themes.
Here are some common examples of themes:
Identifying high level themes is relatively straight forward if the research you are performing has clear goals. Prior to performing the research you may want to explore a certain aspect of the project, for example: participants’ appetite for an online community. Any observations and discussions around this point can then be coded as part of the ‘Community’ theme. Other examples of high level themes might include the navigation or structure of the project.
Themes can also form during the testing process. For example, it may become evident that certain terminology is confusing to the participants. Creating a theme around these issues can assist in identifying these problems and lay the foundations to explore more suitable terminology.
When reviewing your findings you may identify further themes that were not evident when conducting the research. This is where the coding scheme developed earlier can add further value. Reviewing the codes and comments might uncover that there is a common response to a specific question, which didn’t match the observed behaviour. A theme could be created to group these together and this would potentially warrant further investigation.
Below is an example of a coding system I used for a recent user research project. It is essential that the coding system is clear to you and quick to apply during the note taking process.
# - Question number - refer to script
“Quote” - User quote
T1 - Theme 1 - refer to script
T2 - Theme 2 - refer to script
T3 - Theme 3 - refer to script
[O] - Observation
[U] - Usability issue
[Term] - Terminology
[I] - Idea
And here is a page of my research notes with the associated codes applied:
The next time you are conducting user research consider creating your own coding scheme and associated themes. Here are just some of the potential benefits: Memory - writing (particularly with pen and paper) and coding observations helps to embed the findings in your memory. By actively creating associations between notes we are creating extra memory cues. Interpretation - it provides you with structured notes that are much easier to interpret during analysis of the research. Time - saves time as you can minimise time spent re-watching the recordings of the research sessions. Findings - Provides the structure for presentation of the research findings. Additional research - Helps identify areas for further investigation.