Posted by & filed under UX Research.

Recently, the PatternFly team undertook a brief research study to help decide how to handle scrolling within modal dialogs that have more content than can fit within the current view or “above the fold.” In this post, you’ll read about what we were trying to learn, how we constructed the study, and our final conclusions.

What were we trying to learn?

When there is too much content in a modal dialog, how do you expect to find the additional information? Would you expect the buttons to always be visible and what scrolls is a small frame of content? Would you expect to scroll the entire modal dialog to see the buttons at the bottom?  Bootstrap, which PatternFly is based on, uses the latter design by default. Before we recommended one style over another, we wanted to see if there was a user-based need to deviate from the Bootstrap default.

Ultimately, the question we wanted to answer was: “For modal dialogs, which scrolling style should we recommend (and document and build out) in PatternFly?”

How we recreated the issue at hand

To turn this study around quickly, we looked to the Red Hat body of design work to find inspiration that we could riff on. Study participants worked with a live-code prototype that was a derivation of a screen already developed for the Red Hat Cloud Infrastructure installation wizard, which saved us a bunch of time. During the sessions, participants were asked to select the nodes to use for a deployment (in this case, Red Hat OpenStack) and resolve any errors. Once the minimum number of nodes was selected and all errors were cleared up, the Deploy button became enabled and the user could proceed.

Two versions of the prototype were created: one where the entire modal scrolls and the other where the buttons remain in a fixed place and just the content scrolls. Because participants would see and work with both versions of the prototype, to reduce first-impression bias, moderators alternated which version participants saw first.

ModalA

Version A: the entire modal scrolls

 

ModalB

Version B: just a small frame of content scrolls

When a participant selects a node that has multiple interfaces, a message appears indicating that there is an additional step. This message appears below the selected node and shifts the remainder of the list down the page. We included this type of interaction to see if participants would react to some of the nodes “disappearing,” essentially forcing them to scroll, even if, to this point, they weren’t aware they needed to.

Screen Shot 2016-08-04 at 4.42.01 PM

This selected node has multiple network interfaces, and one must be selected before proceeding

How the sessions were conducted

Participants were instructed to select a number of nodes for their deployment. Some of the nodes they needed to select showed the message reminding them to select a network interface as well and some nodes were lower on the modal dialog and out of view. To ensure that participants had to scroll at least a little bit, we raised the minimum number of nodes and instructed them to select some that were “below the fold.”

Session moderators were given a list of prompting questions and things to look out for as the participant worked through the task.  Those included:

  • Did the participant have any reaction to a row disappearing below the fold when an error message appeared?
  • Was it obvious how to find the other nodes?
  • Did the participant have any problems with scrolling?
  • Does the participant correctly identify that the Deploy button becoming enabled as having completed everything they need to?

After the participants have worked with both versions, the moderators asked these questions:

  • How difficult or easy was it to complete this task? The rating scale ranged from 1 (very difficult) to 7 (very easy).
  • Which type of scrolling have you seen in the tools you use daily?
  • Which version was easier?
  • Which did you prefer?
  • Do they have any commentary/preference regarding the order of the Cancel and Deploy buttons?  (We threw this in to see if we could gain any insight on the eternal discussion of whether the OK button should go on the left or the right of the Cancel button.)

What we found out

Overall, participants did not have problems using either version. Only 5% of participants had a reaction to a row disappearing below the fold in either version. No participants had errors finding other nodes, 5% of participants had an error while scrolling on the whole modal scroll (version A), and 5% of participants had an error identifying when the task was complete using either modal version.

Participants rated both modal versions as easy to use and nearly equally easy to use. The whole modal scroll (version A) was rated 5.8 on a 7-point ease of use scale whereas the sticky button modal scroll (version B) was rated 5.9. The low error rate and high and similar ease of use ratings show that it is very unlikely that users will encounter usability issues for either modal that would negatively impact their use of the widget.

When asked for their preference between the two modal versions, participants were split completely down the middle. Slightly more participants rated the whole modal scroll (version A) as easier to use, 45% versus 31%. The remaining participants rated both versions equally easy to use. Most participants did not recall which style modal they were more used to seeing, although of the participants who did, slightly more were familiar with the sticky button modal scroll (version B). There was no evidence of a first impression bias when participants stated which modal version they preferred or which was easier. Additionally, 90% of participants were unable to determine the difference between the two modal versions without prompting from the researcher.

These low error rates, nearly equal ratings, and lack of initial distinction upon seeing the two modal versions imply that participants did not have a strong preference for one modal over the other nor would one version introduce any usability concerns. Since our quantitative analysis didn’t indicate a clear modal version to recommend for PatternFly, we looked to the comments we collected during the testing. Participants tended to talk about the whole modal scroll (version A) more positively, specifically about the fact that more content was visible on the page at one time. Most participants liked the idea of having sticky buttons but some felt it made the content feel too cramped.

And the winner is…

This one is as close to a coin toss as you can get. The two modal versions tested in this study were nearly equal in all aspects. The whole modal scroll (version A) has a slight edge in the comments and poses no usability concerns. For these reasons, we recommend that PatternFly adopts the default Bootstrap default modal style, the whole modal scroll.

The scrolling method is only one factor when deciding how to handle this situation in your own design work. You should also consider the size of the devices that you expect your users to view the UI on and how much data will be shown within the modal. Have you run into this situation before?  How did you handle it?  Let us know your thoughts in the comments.

 

(Many thanks to my co-author and co-moderator Allie Jacobs. Big thanks also to Andrés Galante and Jeff Phillips for creating the excellent prototype, and to session notetakers Mary Clarke, Colleen Hart, and Matt Reid.)

One Response to “How we scroll”

  1. Patrick Riley

    Great article SJ & Allie! The gifs you made describe the issue very well – I find myself liking version B more, but I can definitely see version A being suitable as well. Thanks for sharing this story!
    -Patrick

    Reply

Leave a Reply

(will not be published)