Evaluation of Websites

Why evaluate?

• attract more visitors
• sell more products
• decide which web app to use
• improve visitor ratings
• etc.

Evaluation studies

• always start with a clear research question
  • aka “problem statement”
  • practical/theoretical relevance, feasible
  • types of studies
    • explorative — what is related?
      • e.g. why do people visit the site again?
    • descriptive — what happens?
      • e.g. how many people find the site through a SE?
    • explanatory — why does it happen?
      • if you add login, do people visit the site again?
      • does a change in structure make it easier for visitors to find what they’re looking for?
• make a hypothesis
  • a prediction of outcome of test
  • deduced from theory or observations
• collect data
  • e.g. in lab experiments, survey, interview…
  • qualitative (non-numerical) and quantitative (numerical)
  • test dependent var with respect to independent var
  • specific population (customers/all web users/registered users/whatever)
  • specific sample (random/convenience/volunteers)
• evaluation methods

  • common

    • mockups
      • low fidelity — early in design phase, only basic functionality, static, cheap. focus on concepts.
      • high fidelity — later in design phase, refined details, expensive.
    • prototypes — working example of website
    • focus groups — moderated group discussion, early in design stage.
    • card sorting — group of people sort items into clusters to get intuitive structure for website
    • usability inspection — go systematically through website, check against Ten Web Guidelines. performed by dev team.
    • group walkthrough — group of people walk through website as if performing primary tasks
    • user testing — remote, observe user while primary tasks are performed. log actions, eye tracking, record video/audio.
    • survey

  • specific to web evaluation

    • web analytics
      • analyse logfiles
      • JavaScript page tagging to capture visitor data
      • very good objective data, but privacy concerns and no insight into motivation or unvisited pages.
    • online experiments
      • distribute visitors over versions, see which performs better
      • after release
      • example: A/B testing