Slides Q&A: Slides - How can we improve empirical research on understanding visual information?
.jpg)
Charles Perin
11:27 AM
This panel about empirical research is clearly a panel about quantitative methods for conducting empirical research.
Any advice regarding when and why use quantitative methods and when and why use qualitative methods? Remember this
discussion about precision vs generalizability vs realism this week
19 up votes
0 down votes
Charles Perin
11:27 AM
This panel about empirical research is clearly a panel about quantitative methods for conducting empirical research.
Any advice regarding when and why use quantitative methods and when and why use qualitative methods? Remember this
discussion about precision vs generalizability vs realism this week
19 up votes
0 down votes
.jpg)
Amelia McNamara
10:55 AM
What are the best practices for statistical analysis in the vis field? Should we be doing t-tests? Randomization tests?
Reporting effect sizes? Confidence intervals?
16 up votes
0 down votes
Amelia McNamara
10:55 AM
What are the best practices for statistical analysis in the vis field? Should we be doing t-tests? Randomization tests?
Reporting effect sizes? Confidence intervals?
16 up votes
0 down votes
.jpg)
Chat Wacharamanotham
11:05 AM
Ronald's recommendation is to have multiple hypotheses and multiple conditions, resulting in more complex experimental
designs. Pierre advocates simple experimental design to allow straight-forward statistical analysis and plotting
effect sizes visually. How can we reconcile these recommendations?
11 up votes
0 down votes
Chat Wacharamanotham
11:05 AM
Ronald's recommendation is to have multiple hypotheses and multiple conditions, resulting in more complex experimental
designs. Pierre advocates simple experimental design to allow straight-forward statistical analysis and plotting
effect sizes visually. How can we reconcile these recommendations?
11 up votes
0 down votes
Anonymous
10:43 AM
How can we educate reviewers on basics of cognitive or perceptual techniques, especially in sci vis where user evaluation
is less common
8 up votes
0 down votes
Anonymous
10:43 AM
How can we educate reviewers on basics of cognitive or perceptual techniques, especially in sci vis where user evaluation
is less common
8 up votes
0 down votes
.jpg)
Ben Shneiderman
10:57 AM
I think it is counterproductive to separate Understanding & Application. Kurt Lewin (social psychologist) wisely
said: if you want to understand something, try to change it. I'm tired of researchers hiding behind the "understanding"
wall, when they would do better if they got our & tried to change
8 up votes
0 down votes
Ben Shneiderman
10:57 AM
I think it is counterproductive to separate Understanding & Application. Kurt Lewin (social psychologist) wisely
said: if you want to understand something, try to change it. I'm tired of researchers hiding behind the "understanding"
wall, when they would do better if they got our & tried to change
8 up votes
0 down votes
.jpg)
Pierre Dragicevic
11:10 AM
What are your techniques for not fooling yourself?
8 up votes
0 down votes
Pierre Dragicevic
11:10 AM
What are your techniques for not fooling yourself?
8 up votes
0 down votes
.jpg)
Robert Kosara
11:15 AM
Isn't it amazing that the WHY is not researched more? What does that say about visualization being a science?
7 up votes
0 down votes
Robert Kosara
11:15 AM
Isn't it amazing that the WHY is not researched more? What does that say about visualization being a science?
7 up votes
0 down votes
.jpg)
Steven Franconeri
11:22 AM
What could vision scientists discover that would most impact vis? What problems should we inspire them to go solve?
7 up votes
0 down votes
Steven Franconeri
11:22 AM
What could vision scientists discover that would most impact vis? What problems should we inspire them to go solve?
7 up votes
0 down votes
Pierre Dragicevic
11:09 AM
What vision does *not* tell us about visualization?
6 up votes
0 down votes
Pierre Dragicevic
11:09 AM
What vision does *not* tell us about visualization?
6 up votes
0 down votes
.jpg)
Anamaria Crisan
11:00 AM
P value hacking means missing some really interesting effects in the data because pvalues only tell us how much evidence
we have to support the effect in *our* data - so we miss cool but not significant effects. So I push back, is there
ever really a good time to p-hack?
5 up votes
0 down votes
Anamaria Crisan
11:00 AM
P value hacking means missing some really interesting effects in the data because pvalues only tell us how much evidence
we have to support the effect in *our* data - so we miss cool but not significant effects. So I push back, is there
ever really a good time to p-hack?
5 up votes
0 down votes
Robert Kosara
11:21 AM
Confidence intervals are analogous to p-value testing, but aren't they much more informative? Especially when there
is no significant difference?
5 up votes
0 down votes
Robert Kosara
11:21 AM
Confidence intervals are analogous to p-value testing, but aren't they much more informative? Especially when there
is no significant difference?
5 up votes
0 down votes
.jpg)
Wolfgang Aigner
11:36 AM
What are good starting points for learning how to do Bayesian analysis?
5 up votes
0 down votes
Wolfgang Aigner
11:36 AM
What are good starting points for learning how to do Bayesian analysis?
5 up votes
0 down votes
.jpg)
Michael Correll
11:38 AM
We seem to have a stats problem (wrong stats), a methods problem (wrong studies), a steering problem (wrong research
questions), and a culture problem (wrong incentives). Is there anything that we're doing better than our peers in other
fields?
5 up votes
0 down votes
Michael Correll
11:38 AM
We seem to have a stats problem (wrong stats), a methods problem (wrong studies), a steering problem (wrong research
questions), and a culture problem (wrong incentives). Is there anything that we're doing better than our peers in other
fields?
5 up votes
0 down votes
.jpg)
Karthik Badam
11:40 AM
How do you ensure that the "priors" that you choose are appropriate for your experiment setup? Considering that in VIS
even when we are building on a previously published experiment there might be differences in the visualization design.
Could you provide any resources or suggestions?
5 up votes
0 down votes
Karthik Badam
11:40 AM
How do you ensure that the "priors" that you choose are appropriate for your experiment setup? Considering that in VIS
even when we are building on a previously published experiment there might be differences in the visualization design.
Could you provide any resources or suggestions?
5 up votes
0 down votes
.jpg)
Laura Matzen
11:17 AM
What are some of the top research questions that visualization researchers want cognitive scientists/vision scientists
to answer? In other words, what kinds of information about vision and cognition would be most useful for the visualization
field?
4 up votes
0 down votes
Laura Matzen
11:17 AM
What are some of the top research questions that visualization researchers want cognitive scientists/vision scientists
to answer? In other words, what kinds of information about vision and cognition would be most useful for the visualization
field?
4 up votes
0 down votes
Anonymous
11:32 AM
How can we prevent against prior-hacking in bayesian statistics? :)
4 up votes
0 down votes
Anonymous
11:32 AM
How can we prevent against prior-hacking in bayesian statistics? :)
4 up votes
0 down votes
Shaun Kurian
10:38 AM
What is the weight significance of eye tracking as opposed to user evaluation surveys and interviews? Could eye tracking
offset the lack of numbers in usability studies and provide a more empirical value?
2 up votes
0 down votes
Shaun Kurian
10:38 AM
What is the weight significance of eye tracking as opposed to user evaluation surveys and interviews? Could eye tracking
offset the lack of numbers in usability studies and provide a more empirical value?
2 up votes
0 down votes
.jpg)
Eamonn Maguire
10:43 AM
Even though we have a simple visual encoding in English (an alphabet), the ambiguity in how we extract information from
a text is often amazing. What can we learn from the humanities in this respect.
2 up votes
0 down votes
Eamonn Maguire
10:43 AM
Even though we have a simple visual encoding in English (an alphabet), the ambiguity in how we extract information from
a text is often amazing. What can we learn from the humanities in this respect.
2 up votes
0 down votes
Anonymous
11:03 AM
How to measure "How creative / supervise / inspired" subjects are when they use my visual analysis tools. And how to
compare it with alternatives?
2 up votes
0 down votes
Anonymous
11:03 AM
How to measure "How creative / supervise / inspired" subjects are when they use my visual analysis tools. And how to
compare it with alternatives?
2 up votes
0 down votes
Anonymous
11:36 AM
Infoviz studies need to measure performance metrics as well as user perception and experience metrics. But a lot of
current rigorous statistical analysis are more applicable to performance metrics rather than user perception and user
reported metrics. So how do we resolve that? Should we even try?
2 up votes
0 down votes
Anonymous
11:36 AM
Infoviz studies need to measure performance metrics as well as user perception and experience metrics. But a lot of
current rigorous statistical analysis are more applicable to performance metrics rather than user perception and user
reported metrics. So how do we resolve that? Should we even try?
2 up votes
0 down votes
Eamonn Maguire
11:38 AM
I would argue that using old techniques is not a bad thing. Using old techniques badly is the problem. P values are
fine, if only people didn't apply them blindly.
2 up votes
0 down votes
Eamonn Maguire
11:38 AM
I would argue that using old techniques is not a bad thing. Using old techniques badly is the problem. P values are
fine, if only people didn't apply them blindly.
2 up votes
0 down votes
.jpg)
Geoffrey Ellis
11:25 AM
How do we find Ronald's 'fruit flies' in the context of participants for empirical studies?
1 up vote
0 down votes
Geoffrey Ellis
11:25 AM
How do we find Ronald's 'fruit flies' in the context of participants for empirical studies?
1 up vote
0 down votes
.jpg)
Jessica Hullman
11:32 AM
Pierre's biovis keynote on visualizing results: https://www.lri.fr/~dragice/statistical-dances/assets/player/KeynoteDHTMLPlayer.html
Description: http://aviz.fr/badstats#sec0 Matt's work on why bayesian stats better matches HCI culture and incentives:
http://www.greglnelson.info/chi_2016_bayes.pdf
1 up vote
0 down votes
Jessica Hullman
11:32 AM
Pierre's biovis keynote on visualizing results: https://www.lri.fr/~dragice/statistical-dances/assets/player/KeynoteDHTMLPlayer.html
Description: http://aviz.fr/badstats#sec0 Matt's work on why bayesian stats better matches HCI culture and incentives:
http://www.greglnelson.info/chi_2016_bayes.pdf
1 up vote
0 down votes
Anonymous
11:38 AM
Need a few good resources for students. Where are they?
1 up vote
0 down votes
Anonymous
11:38 AM
Need a few good resources for students. Where are they?
1 up vote
0 down votes
.jpg)
Sam Quinan
11:45 AM
How confident can we really be about the generalizability of identified effects across populations? Specifically, to
what extent should or shouldn't we be expecting effect differences between the general public and a given community
of domain experts?
1 up vote
0 down votes
Sam Quinan
11:45 AM
How confident can we really be about the generalizability of identified effects across populations? Specifically, to
what extent should or shouldn't we be expecting effect differences between the general public and a given community
of domain experts?
1 up vote
0 down votes
Sara Borthwick
10:47 AM
With Microsoft's announcement yesterday that Paint will support 3D and AR headsets will start at $300, making 3D visuals
accessible to mass market and non-Vis audiences, do you think that there will be boom in 3D infovis research? What
would be the big questions?
0 up votes
0 down votes
Sara Borthwick
10:47 AM
With Microsoft's announcement yesterday that Paint will support 3D and AR headsets will start at $300, making 3D visuals
accessible to mass market and non-Vis audiences, do you think that there will be boom in 3D infovis research? What
would be the big questions?
0 up votes
0 down votes
Bilal A
11:11 AM
Can we learn underlying principles by studying perception in 5-year old humans? Or animals? Do they have cognitive bias?
0 up votes
0 down votes
Bilal A
11:11 AM
Can we learn underlying principles by studying perception in 5-year old humans? Or animals? Do they have cognitive bias?
0 up votes
0 down votes
Geoffrey Ellis
11:19 AM
Where do we find Ronald's 'fruit flies' in the context of participants for studies? If you can offer grade points to
your students then great but that's a luxury to many of us have to make use of AMT with all its problems and cost.
0 up votes
0 down votes
Geoffrey Ellis
11:19 AM
Where do we find Ronald's 'fruit flies' in the context of participants for studies? If you can offer grade points to
your students then great but that's a luxury to many of us have to make use of AMT with all its problems and cost.
0 up votes
0 down votes
Anonymous
11:52 AM
This discussion should be further had and explored in our respective labs. Will the slides be available somewhere for
future support ?
0 up votes
0 down votes
Anonymous
11:52 AM
This discussion should be further had and explored in our respective labs. Will the slides be available somewhere for
future support ?
0 up votes
0 down votes
