Author Archives: Steve Haroz

Guide to user performance evaluation at InfoVis 2017

Previous years: 2013, 2014, 2015, 2016

The goal of this guide is to highlight vis papers that demonstrate evidence of a user performance benefit. I used two criteria:

  1. The paper includes an experiment measuring user performance (e.g. accuracy or speed)
  2. Analysis of statistical differences determined whether results were reliable.

I did not discriminate beyond those two criteria. However, I am using a gold star to highlight one property that only a few papers have: a generalizable explanation for why the results occurred. You can read more about explanatory hypotheses here.

Continue reading

by Leave a comment

Open Access VIS

The purpose of Open Access Vis is to highlight open access papers, materials, and data and to see how many papers are unavailable outside of a paywall. See the about page for more details about reliable open access.

Why?

Most visualization research papers are funded by the public, reviewed and edited by volunteers, and formatted by the authors. So for IEEE to charge $33 for each person who wants to read the paper is… well… (I’ll let you fill in the blank). This paywall is contrary to the supposedly public good of research and the claim that visualization research helps practitioners (who are not on a university campus).

But there’s an up side. IEEE specifically allows authors to post their version of a paper (not the IEEE version with a header and page numbers) to:

  • The author’s website
  • The institution’s website (e.g., lab site or university site)
  • A pre-print repository (which gives it a static URL and avoids “link rot”)

Badges

Continue reading

by 2 Comments

Guide to user performance evaluation at InfoVis 2016

Previous years: 2013, 2014, 2015

The goal of this guide is to highlight vis papers that demonstrate evidence of a user performance benefit. I used two criteria:

  1. The paper includes an experiment measuring user performance (e.g. accuracy or speed)
  2. Analysis of statistical differences determined whether results were reliable.

I did not discriminate beyond those two criteria. However, I am using a gold star to highlight one property that only a few papers have: a generalizable explanation for why the results occurred. You can read more about explanatory hypotheses here.

Continue reading

by

A Look at the Keywords in InfoVis 2016 Submissions

It’s that time of year again. InfoVis abstracts have been submitted, and lots of people are scrambling to finish their full submission.

I was curious about the distribution of keywords in the submissions, so I visualized some of the data available to the program committee (PC). After checking with the chairs, I thought others might be curious about the results.

Note that these are only abstracts, so there will probably be some attrition before the full paper submission deadline. To see a MUCH more thorough analysis of multiple years and venues, check out http://keyvis.org

Continue reading

, by

Guide to user performance evaluation at InfoVis 2015

The goal of this guide is to highlight vis papers that demonstrate evidence of a user performance benefit. I used two criteria:

  1. The paper includes an experiment measuring user performance (e.g. accuracy or speed)
  2. Analysis of statistical differences determined whether results were reliable.

I did not discriminate beyond those two criteria. However, I am using a gold star to highlight one property that only a few papers have: a generalizable explanation for why the results occurred. You can read more about explanatory hypotheses here.

Continue reading

by

3 Tips for Hiding Your Research Publications

As we all know, the most important part of publishing research is making sure that no one ever reads or cites it. After all, it’d be awful if anyone actually saw the end result of months or even years of your effort. So keep these tips in mind the next time you author a publication.
Continue reading

, by
citation not always needed (via XKCD)

Why I Don’t Write a “Related Work” Section

Imagine someone explaining a complex topic, like how to improve the fuel efficiency of a boat. But shortly after starting the explanation, they go off on a series of tangents about pretty boats they’ve seen, big boats, rubber ducks, submarines, and other transportation vehicles such as the new 787 by Boeing et al. Then they return to the explanation of boat efficiency without ever referencing why they brought up those strange tangents.

Tangents are confusing, and they hurt clarity. The related work section is often just a string of unrelated tangents, which is a waste of the reader’s time.

Now let me make something clear: I am not necessarily saying that papers should cite fewer sources. Instead, each citation should serve an obvious, specific purpose. And if that purpose is so tangential to the structure of your argument that you need to put it in what amounts to a citation dumping ground, then it isn’t needed.

What’s the purpose of a citation?

Continue reading

Posted in CHI, Science, Visualization by .

InfoVis 2014 – The methods papers

This year, 40% of InfoVis papers included an empirical evaluation. I made a list in my last post.

There were also a couple papers worth noting that described methods for evaluating visualizations. These papers can help bootstrap future evaluations, leading to a better understanding of when and why vis techniques are effective.

Learning Perceptual Kernels for Visualization Design – Çağatay Demiralp, Michael Bernstein, Jeffrey Heer pdf
A collection of methods are described to find the relative discriminability of feature values (e.g. colors or shapes). It also looks at finding the descriminability of combinations of visual features (e.g. colors and shapes). The paper validates its approach by determining the discriminability of size and showing which of their measures closely match the established Steven’s power law for size.

A Principled Way of Assessing Visualization Literacy – Jeremy Boy, Ronald Rensink, Enrico Bertini, Jean-Daniel Fekete pdf
This paper describes how to use Item Response Theory – a technique common in psychometrics and education literature – to assess a person’s “literacy” or skill with visualizations. I would have liked to have seen the approach validated (or at least compared) with some external factor like the person’s experience with visualization. Understandably, that can be tough to measure, but this method certainly shows promise for explaining individual differences in user performance.

by

Guide to user performance evaluation at InfoVis 2014

The goal of this guide is to help highlight papers that demonstrate evidence of a user performance benefit. I used two criteria:

  1. The paper includes an experiment measuring user performance (e.g. accuracy or speed)
  2. Analysis of statistical differences determined whether results were reliable.

I did not discriminate beyond those two criteria. However, I am using a gold star to highlight one property that only a few papers have: a generalizable explanation for why the results occurred. You can read more about explanatory hypotheses here.

Continue reading

by

So much we don’t know about visualization

It’s always amazing how many basic visualization questions are yet to be answered. Robert Kosara raised one yesterday: What is the most effective way to show large scale differences?

Rather than using a bar chart to represent values, he made a demo that sequentially shows dots to demonstrate how many more times a CEO makes than a worker. His solution looked compelling, but I realized that I don’t know of any literature in vis that has empirically tackled this problem. A goal as simple as visualizing a pair of values of very different scale has few (if any) guidelines.

Furthermore, although there have been a few papers on animation in charts (e.g. [2, 4]), the basic approach of using animation to represent a single value still has many unanswered questions.

Robert’s demo used both numerocity and duration of the animation to visualize each value. I forked his code to make a demo of some alternative animation styles (options at the bottom), but I don’t know of any literature that hints if or why one would be better than another:

Continue reading

, by