{"id":513,"date":"2015-10-24T10:00:29","date_gmt":"2015-10-24T15:00:29","guid":{"rendered":"http:\/\/steveharoz.com\/blog\/?p=513"},"modified":"2015-10-24T10:38:11","modified_gmt":"2015-10-24T15:38:11","slug":"guide-to-user-performance-evaluation-at-infovis-2015","status":"publish","type":"post","link":"http:\/\/steveharoz.com\/blog\/2015\/guide-to-user-performance-evaluation-at-infovis-2015\/","title":{"rendered":"Guide to user performance evaluation at InfoVis 2015"},"content":{"rendered":"<p>The goal of this guide is to highlight vis papers that demonstrate\u00a0evidence of a user performance benefit. I used two criteria:<\/p>\n<ol>\n<li>The paper includes an\u00a0experiment measuring user performance (e.g. accuracy or speed)<\/li>\n<li>Analysis of statistical differences determined whether\u00a0results were reliable.<\/li>\n<\/ol>\n<p>I\u00a0did not discriminate beyond those two criteria. However, I am using a gold star <span style=\"color: #ffcc00; font-size: 1.15rem; line-height: 100%;\">\u2605<\/span> to highlight one property\u00a0that only a few papers have: a generalizable explanation for why the results occurred. You can read more about explanatory hypotheses <a title=\"Mysterious Origins of Hypotheses in Visualization and CHI\" href=\"http:\/\/steveharoz.com\/blog\/?p=304\">here<\/a>.<\/p>\n<p><!--more--><br \/>\n<strong>A Psychophysical Investigation of Size as a Physical Variable<\/strong> &#8211; Yvonne Jansen, Kasper Hornbaek.<br \/>\n<a id=\"6a2c8de7bb113\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205993?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205993?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb113\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/yvonnejansen.me\/articles\/VIS2015%20-%20size%20as%20physical%20variable.pdf\">pdf<\/a><\/p>\n<p><strong><span class=\"star\"> <a title=\"Explanatory\" href=\"http:\/\/steveharoz.com\/blog\/?p=304\">Explanatory <\/a><\/span>Beyond Memorability: Visualization Recognition and Recall<\/strong> &#8211; Michelle A. Borkin, Zoya Bylinskii, Nam Wook Kim, Constance May Bainbridge, Chelsea S. Yeh, Daniel Borkin, Hanspeter Pfister, Aude Oliva.<br \/>\n<a id=\"6a2c8de7bb16b\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/137219625?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/137219625?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb16b\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/vcg.seas.harvard.edu\/files\/pfister\/files\/infovis_submission251-camera.pdf\">pdf<\/a><\/p>\n<p><strong><span class=\"star\"> <a title=\"Explanatory\" href=\"http:\/\/steveharoz.com\/blog\/?p=304\">Explanatory <\/a><\/span> Beyond Weber&#8217;s Law: A Second Look at Ranking Visualizations of Correlation<\/strong> &#8211; Matthew Kay, Jeffrey Heer.<br \/>\n<a id=\"6a2c8de7bb19d\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205873?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205873?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb19d\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/idl.cs.washington.edu\/files\/2015-BeyondWebersLaw-InfoVis.pdf\">pdf<\/a><\/p>\n<p><strong>Comparing Color and Leader Line Highlighting Strategies in Coordinated View Geovisualizations<\/strong> &#8211; Amy L. Griffin, Anthony C. Robinson.<br \/>\n<a id=\"6a2c8de7bb1c9\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136210730?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136210730?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb1c9\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"https:\/\/www.researchgate.net\/profile\/Anthony_Robinson6\/publication\/271964694_Comparing_Color_and_Leader_Line_Highlighting_Strategies_in_Coordinated_View_Geovisualizations\/links\/5511694e0cf24e9311ce4482.pdf\">pdf<\/a><\/p>\n<p><strong>Guidelines for Effective Usage of Text Highlighting Techniques<\/strong> &#8211; Hendrik Strobelt, Daniela Oelke, Bum Chul Kwon, Tobias Schreck, Hanspeter Pfister.<br \/>\n<a id=\"6a2c8de7bb1f1\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136278904?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136278904?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb1f1\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/textanno.hs8.de\/img\/preprint_textanno_strobelt.pdf\">pdf<\/a><\/p>\n<p><strong> HOLA: Human-like Orthogonal Network Layout<\/strong> &#8211; Steve Kieffer, Tim Dwyer, Kim Marriott, Michael Wybrow.<br \/>\n<a id=\"6a2c8de7bb218\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205841?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205841?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb218\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/marvl.infotech.monash.edu\/~dwyer\/papers\/hola2015.pdf\">pdf<\/a><\/p>\n<p><strong>Improving Bayesian Reasoning: The Effects of Phrasing, Visualization, and Spatial Ability<\/strong> &#8211; Alvitta Ottley, Evan M. Peck, Lane T. Harrison, Daniel Afergan, Caroline Ziemkiewicz, Holly A. Taylor, Paul K. J. Han, Remco Chang.<br \/>\n<a id=\"6a2c8de7bb24b\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205947?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205947?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb24b\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/www.eg.bucknell.edu\/~emp017\/papers\/bayes_infovis2015.pdf\">pdf<\/a><\/p>\n<p><strong>Off the Radar: Comparative Evaluation of Radial Visualization Solutions for Composite Indicators<\/strong> &#8211; Yael Albo, Joel Lanir, Peter Bak, Sheizaf Rafaeli.<br \/>\n<a id=\"6a2c8de7bb275\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205811?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205811?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb275\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <span style=\"color: #b22;\">[This publication is hidden]<\/span><\/p>\n<p><strong>Orientation-Enhanced Parallel Coordinate Plots<\/strong> &#8211; Renata Raidou, Martin Eisemann, Marcel Breeuwer, Elmar Eisemann, Anna Vilanova.<br \/>\n<a id=\"6a2c8de7bb299\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205977?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205977?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb299\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"https:\/\/graphics.tudelft.nl\/Publications-new\/2015\/REBEV15\/OPCPs_337.pdf\">pdf<\/a><\/p>\n<p><strong>Perception-based Evaluation of Projection Methods for Multidimensional Data Visualization<\/strong> &#8211; Ronak Etemadpour, Robson Motta, Jose Gustavo de Souza Paiva, Rosane Minghim, Maria Cristina Ferreira de Oliveira, Lars Linsen.<br \/>\n<a id=\"6a2c8de7bb2c0\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136210739?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136210739?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb2c0\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <span style=\"color: #b22;\">[This publication is hidden]<\/span><\/p>\n<p><strong>Representing Uncertainty in Graph Edges: An Evaluation of Paired Visual Variables<\/strong> &#8211; Hua Guo, Jeff Huang, David H. Laidlaw.<br \/>\n<a id=\"6a2c8de7bb2e4\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136210879?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136210879?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb2e4\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/vis.cs.brown.edu\/docs\/pdf\/Guo-2015-RUG.pdf\">pdf<\/a><\/p>\n<p><strong>SchemeLens: A Content-Aware Vector-Based Fisheye Technique for Navigating Large Systems Diagrams<\/strong> &#8211; Aurelie Cohe, Bastien Liutkus, Gilles Bailly, James Eagan, Eric Lecolinet.<br \/>\n<a id=\"6a2c8de7bb307\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136146638?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136146638?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb307\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/www.gillesbailly.fr\/publis\/BAILLY_SchemeLens.pdf\">pdf<\/a><\/p>\n<p><strong><span class=\"star\"> <a title=\"Explanatory\" href=\"http:\/\/steveharoz.com\/blog\/?p=304\">Explanatory <\/a><\/span>Spatial Reasoning and Data Displays<\/strong> &#8211; Susan VanderPlas, Heike Hofmann.<br \/>\n<a id=\"6a2c8de7bb32e\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205737?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205737?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb32e\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <span style=\"color: #b22;\">[This publication is hidden]<\/span><\/p>\n<p><strong>Suggested Interactivity: Seeking Perceived Affordances for Information Visualization<\/strong> &#8211; Jeremy Boy, Louis Eveillard, Francoise Detienne, Jean-Daniel Fekete.<br \/>\n<a id=\"6a2c8de7bb35c\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205747?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205747?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb35c\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"https:\/\/hal.inria.fr\/hal-01188973\/document\">pdf<\/a><\/p>\n<p><strong>TimeNotes: A Study on Effective Chart Visualization and Interaction Techniques for Time-Series Data<\/strong> &#8211; James Walker, Rita Borgo, Mark Jones.<br \/>\n<a id=\"6a2c8de7bb37f\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205907?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205907?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb37f\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/cs.swan.ac.uk\/~csmark\/PDFS\/2015_TimeNotes.pdf\">pdf<\/a><\/p>\n<p><strong>VectorLens: Angular Selection of Curves within 2D Dense Visualizations<\/strong> &#8211; Maxine Dumas, Michael McGuffin, Patrick Chasse.<br \/>\n<a id=\"6a2c8de7bb3a5\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136210748?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136210748?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb3a5\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/www.maximedumas.ca\/files\/dumas-tvcg2014-VectorLens.pdf\">pdf<\/a><\/p>\n<p><strong>Visual Encodings of Temporal Uncertainty: A Comparative User Study<\/strong> &#8211; Theresia Gschwandtner, Markus Bogl, Paolo Federico, Silvia Miksch.<br \/>\n<a id=\"6a2c8de7bb3c7\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205918?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205918?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb3c7\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <span style=\"color: #b22;\">[This publication is hidden]<\/span><\/p>\n<p><strong>Voyager: Exploratory Analysis via Faceted Browsing of Visualization Recommendations<\/strong> &#8211; Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay, Bill Howe, Jeffrey Heer.<br \/>\n<a id=\"6a2c8de7bb3ea\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136205686?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136205686?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb3ea\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <a href=\"http:\/\/idl.cs.washington.edu\/files\/2015-Voyager-InfoVis.pdf\">pdf<\/a><\/p>\n<p><strong>UnTangle Map: Visual Analysis of Probabilistic Multi-Label Data<\/strong> &#8211; Nan Cao, Yu-Ru Lin, David Gotz.<br \/>\n<a id=\"6a2c8de7bb40b\" rel=\"wp-video-lightbox\" href=\"https:\/\/vimeo.com\/136206607?width=1920&#038;height=1080\" title=\"\">video<\/a>    <script>\r\n    \/* <![CDATA[ *\/\r\n    jQuery(document).ready(function($){\r\n        $(function(){\r\n            var width = $(window).innerWidth();\r\n            var setwidth = parseFloat(1920);\r\n            var ratio = parseFloat(0.5625);\r\n            var height = parseFloat(1080);\r\n            var link = 'https:\/\/vimeo.com\/136206607?width=1920&amp;height=1080';\r\n            if(width < setwidth)\r\n            {\r\n                height = Math.floor(width * 0.5625);\r\n                \/\/console.log(\"device width \"+width+\", set width \"+1920+\", ratio \"+0.5625+\", new height \"+ height);\r\n                var new_url = wpvl_paramReplace('width', link, width);\r\n                var new_url = wpvl_paramReplace('height', new_url, height);\r\n                $(\"a#6a2c8de7bb40b\").attr('href', new_url);\r\n                \/\/console.log(new_url);\r\n            }\r\n        });\r\n    });\r\n    \/* ]]> *\/\r\n    <\/script> <span style=\"color: #b22;\">[This publication is hidden]<\/span><\/p>\n<h2>No Clear Change<\/h2>\n<p><strong>37%<\/strong> of InfoVis conference papers measured user performance. <a href=\"http:\/\/steveharoz.com\/blog\/2014\/guide-to-user-performance-evaluation-at-infovis-2014\/\">Last year<\/a>, I thought there was a big change, but\u00a0the variance is indistinguishable from noise.<\/p>\n<p>Here&#8217;s\u00a0the Aggresti-Coull binomial 84% CI, so each proportion can be compared.<\/p>\n<p><a href=\"http:\/\/steveh.co\/blog\/wp-content\/uploads\/2015\/09\/stats1.svg\"><img decoding=\"async\" class=\"style-svg\" style=\"width: 100%;\" src=\"http:\/\/steveh.co\/blog\/wp-content\/uploads\/2015\/09\/stats1.svg\" alt=\"stats\" \/><\/a><\/p>\n<h2>The Journal\u00a0Articles<\/h2>\n<p>It&#8217;s great that the\u00a0distinction between conference papers and journal papers\u00a0is fading away at least in terms of the conference program. I&#8217;m only maintaining it for historical comparison.<\/p>\n<p>In the chart on the right, I collapsed the past three years of data and recomputed the means and CIs. For TVCG, I&#8217;m only include papers presented under the InfoVis track.\u00a0TVCG has more papers with performance evaluation, and it can&#8217;t simply be explained by random noise. I don&#8217;t know if the difference is caused by where papers are submitted, the different review process, different reviewers, or rolling submissions being more conducive to running a study. But more of the journal papers test their claims.<\/p>\n<h2>Honorable Mentions<\/h2>\n<p>In the last couple years, not a single best paper or honorable mention attempted to experimentally validate their claims. This year changed that trend with awards going to papers that measured (or reanalyzed) user performance.<\/p>\n<h2>Little\u00a0Generalization<\/h2>\n<p>There is still a very low proportion of papers with an explanatory hypothesis that can inform\u00a0generalizability. I try to be very generous with this assessment, but very few papers attempt to explain why or if the results are applicable outside of the specific conditions of the study. Also, there are still a lot of guesses presented as hypotheses.<\/p>\n<p><em>Also see the guides for\u00a0<a href=\"http:\/\/steveharoz.com\/blog\/2014\/guide-to-user-performance-evaluation-at-infovis-2014\/\">2014<\/a> and <a href=\"http:\/\/steveharoz.com\/blog\/2013\/guide-to-user-performance-evaluation-at-infovis-2013\/\">2013<\/a>.<\/em><\/p>\n<p><em>Obviously, please let me know if you find a mistake or think I missed something. Also, please hassle any authors who didn\u2019t make their pdf publicly available.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The goal of this guide is to highlight vis papers that demonstrate\u00a0evidence of a user performance benefit. I used two criteria: The paper includes an\u00a0experiment measuring user performance (e.g. accuracy or speed) Analysis of statistical differences determined whether\u00a0results were reliable. I\u00a0did not discriminate beyond those two criteria. However, I am using a gold star \u2605 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":547,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[4],"tags":[15],"class_list":["post-513","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-science","tag-infovis"],"_links":{"self":[{"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/posts\/513","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/comments?post=513"}],"version-history":[{"count":21,"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/posts\/513\/revisions"}],"predecessor-version":[{"id":594,"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/posts\/513\/revisions\/594"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/media\/547"}],"wp:attachment":[{"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/media?parent=513"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/categories?post=513"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/steveharoz.com\/blog\/wp-json\/wp\/v2\/tags?post=513"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}