Usability of — Performance Problems Trump Effective Design

Image for Usability of — Performance Problems Trump Effective Design

November 19, 2014 update: see our updated review of

Perhaps not since the infamous Florida Butterfly Ballot of 2000 have interaction design and politics come together so publicly.  Last week’s official launch of the Affordable Care Act’s Health Insurance Marketplace websites provided many Americans with their first hands-on experience with the controversial legislation.

Given the lack of understanding and misinformation about the Affordable Care Act, a clear and easy registration process is vital to its success going forward, or as Businessweek  put it, “the surest way to failure is if Americans have a crummy experience when they sign up for it.”

Usability of Obamacare

Creating effective, usable websites for the public requires understanding and accommodating for a wide range of characteristics including user goals, technology constraints, and reading abilities. Throw in the complexities of insurance, healthcare, and multiple languages, and you’ve got a significant design and usability challenge.

As a design firm we frequently conduct expert assessments of websites to identify opportunities for improvement in areas such as efficiency, clarity, and readability. Towards that end we conducted a preliminary evaluation of the site, focusing on the registration process. Our goal was to evaluate the user experience to provide the public (and the site designers themselves) with constructive feedback.

As you may have heard by now, there have been many performance and technical challenges with the site, which is comparable with our own findings. While we found that most aspects of the site were well thought out and designed, our inability to complete the registration process in a timely manner stood out as a significant obstacle to the site’s effectiveness.

Besides the obvious technical performance improvements we recommend that the site is upfront in communicating the ongoing problems, and recommended short-term alternatives.

What We Looked At

We performed an analysis of and attempted to complete the registration process to enable us to login and see the insurance options. The site is fully responsive across device forms and, from a visual perspective, provides a clean, easy-to-navigate experience for the user. Navigation options are appropriately few and calls to action are clear – users can either learn about their options or apply. Key dates (enrollment start/end dates and potential coverage start date) are prominently displayed, as well as a countdown timer for the enrollment end date.

Usability of Obamacare

Other than the hero image (large photograph) at the top of the home page, the color scheme for most of the site is a pleasant mix of white, gray, and blues. This provides adequate distinction between sections, easily identifies links and calls-to-action, and limits potential accessibility issues for those with color blindness. The site is available in Spanish, as indicated by the traditionally placed Español link in the top right corner.  Although help information is also available for many speaking other languages, the link to that help is at the bottom of the page, not visible without scrolling.

Usability of Obamacare

As many of the intended audience for this site may not be English-speakers, a more prominent display would be beneficial.

When learning about options, users can either explore on their own through a well-organized set of topics and frequently asked questions, or they can be guided through a short set of questions that will help identify applicable options, provide a targeted set of help/information links, and suggest a set of “next steps.” These steps include a link to a specific state-run exchange if the Federal exchange is not appropriate for the user. Interestingly, before the October 1 start date, the series of questions included asking the user’s gender – this is no longer the case and is appropriate given that gender would have no impact on the information returned. The data are segmented well, preventing a text barrage that could be possible given the amount of information the site needs to provide.

In an attempt to get feedback on the site from its users, just about every page has a “Was this helpful?” survey link to provide positive or negative feedback.

Usability of Obamacare

This is an easy way for the Government to obtain information to improve the site; however the placement of the link doesn’t always make it clear whether one is responding regarding the entire page, a specific section on the page, or a specific piece of information. Just in the few days we were looking at the site the placement of the links was adjusted slightly to help distinguish it a bit.

Although a user may access all of the general information and educational resources without an account, one must create an account in order to access the detailed insurance information, such as premiums and deductibles. The site uses a typical registration approach, requiring the user to create an ID, password (re-entered to ensure it is correct), and selecting and providing answers to three security questions. After completing the information, a confirmation email is delivered to the user, who then must click a link in the email to confirm the registration. Field requirements (e.g., password length and characters) are well identified and the registration process is clean.

Performance Issues

Although we were overall very impressed with the site usability in terms of workflow, communication, and visual design, the technical issues we encountered actually trying to use the site easily overrode those successes.  Presumably due to volume of use, poor code, incorrect error messaging, hardware malfunction, or a combination, we were never able to actually get to the point where we could access the insurance information.

Volume of usage was likely the culprit for trying to create an account, as multiple attempts simply resulted in a notice that due to the number of visitors, we were put into a queue and would be directed to the login page when possible. It was clear that the site was being worked on live, as the notifications changed during our testing:

Usability of Obamacare

Unfortunately, there is no indicator of how long this may take nor a user’s place in the queue. This will likely not be an issue once the ACA is fully established and fewer people are hitting the site at once, but it makes for a frustrating experience now.

Once we made it into the registration process, we ran into issues when we reached the security questions section. Although the list of questions did initially appear, they disappeared after inadvertently tabbing off of the first list and would not reappear. Starting the registration process over did not correct the issue and it wasn’t until we switched our residence state that it seemed to recover. It may have been a cookie issue and the state switch eliminated that, but completing the process until then was not possible.

Getting past this hurdle was not the end to our issues, however, as our first attempt at clicking the final Create Account button resulted in:

Usability of Obamacare

Clicking Try Again put us back into the queue to get to the login page.

Eventually we got a Success notification page; the final step was to click the link provided in our confirmation email message. Goal in sight, we clicked:

Usability of Obamacare

The next day we managed to get past this glitch and reached the login page. After several attempts at logging in and receiving an error indicating the entered information wasn’t valid, we thought maybe we hadn’t created the password we thought we did and started the password reset process. We entered our account ID, which generated an automated email to begin the reset.  Unfortunately clicking that link resulted in:

Usability of Obamacare

The email we received was generated solely by use entering our account ID, not email address. If they were able to send us an email it meant there is a valid account somewhere in the system. We tried several other times to reset the password with the same result. Unable to login, we started the registration process over and created a new account.

Either we were lucky or the coders had corrected found errors, as creating the second account went much more smoothly. Logging in with the new account seemed to be working, as the page title changed to “success URL”. Unfortunately the page never loaded and we halted our efforts at that point.

Takeaways and Next Steps

Although initially using the site was a positive experience, the fundamental challenge with completing the registration and login processes prevent it from being successful. A six-month lead-time in enrollment may ease the pressure on enrollment, but it’s little consolation to frustrated users. This situation is likely exacerbated for infrequent internet users, those leery of technology, and/or those who may approach Obamacare with distrust.

In summary, we found:


  • does a very good job of incorporating visual hierarchy, colors, and few, but appropriate, navigation options to create an appealing design.
  • Information and help topics are well-organized and provide contextual suggestions for related content. Additional help is available via a series of YouTube videos, as well as phone and TTY access.
  • It is clear that site support is very actively working, as we noticed several changes during our short time using the site. For example, progress notifications and error messages became more specific and applicable.


  • Even the most user-friendly site fails when technical issues preclude users from achieving goals. The numerous registration and login issues prevent the site from being successful.  Fixing these issues should take priority over any of the other noted issues.

This site could be well served to set better expectations by communicating known issues and delays saliently on the home page. In addition to potentially saving a user from starting a process they won’t be able to complete, it let’s the visitors know someone is aware of the issue and it is being addressed. This can go a long way in building trust and satisfaction with the users.

We’ll plan to revisit and also take a look at the site in future weeks when (hopefully) the registration/login process can be effectively completed.

Illustration by Morgan Knepper

About the Author

Image of Steve Jones

Steve Jones

Senior Researcher

Steve's main goal has been to make sure that a user’s response would never be, “Why did they put that over there?” More