2008年11月29日土曜日
2008年11月15日土曜日
KING MIDAS HAS DONKEY'S EARS
from http://www.stefanmart.de/13_midas/130e_midas.htm
powered by http://www.ispeech.org/
2008年11月3日月曜日
2008年10月13日月曜日
2008年10月11日土曜日
2008年10月6日月曜日
2008年10月5日日曜日
2008年9月27日土曜日
2008年9月26日金曜日
2008年9月21日日曜日
2008年9月13日土曜日
2008年9月7日日曜日
2008年8月30日土曜日
Wikis And Widgets And Blogs Oh My!
Wikis And Widgets And Blogs Oh My!
View SlideShare presentation or Upload your own.
全体の投票数が15しかないので、はじめの方の統計は鵜呑みにはできないが、タイトルからして、Wiki と Widget と Blog に重要性を置いていると言えるだろう。オリジナルのファイルはダウンロード可能。ノートに多少書き込みがある。大きな画像を使っているので、ファイルのサイズも 29.2 mb と、かなりでかい。
2008年8月24日日曜日
SHOCK
Read this document on Scribd: SHOCK
My great friends let this not come to you as 網友你好 希 望這 份檔案 不致 造成你 的 a surprise, 驚訝 but it's real 內容都 是真實事 件 have them living around us and 現在就 發生在 in our neighbourhood today, 我們週 遭 we can change it with prayers, and 大家可 以用祈 禱來 幫助他 們 always lending a helping hand to those in need. 更重 要的 是伸 出援手 給他們實 質的救 助 Don't keep this email to yourself, 看到本 文 不要留在身邊 forward it to your friends, so our friends and all people will 轉寄給 你朋友們 讓大 家都能 thank NATURE 感謝上蒼 for food and water that they already have. 讓我 們享 有食物 和飲 水 This is one more reason 另外 why we have to thank God for the food that 我們要 感激上 蒼 we can have easily… 因為我 們要 有食物 實 在太 容易啦 But in the other hand... ironically, 所以 we still waste the food that we 我們不可 以糟 蹋食物 buy. I feel very GRATEFUL 我們應 該感恩 for what I have today... 對於現在 所有 的一切 We are so Blessed for the wonderful works of 因為 我們 深受上 蒼賜 福 NATURE's hand in our life today, 以及 大自然母親 的照料 just think of this.... 原因 是 看看下面 .... "I felt very fortunate to live in this part of the world. 我們應 該要為 能活 在現世 而感 到幸 福 I promise I will never waste my food no matter 不管怎樣 我們以後 不要再浪費 食物 how bad it can taste and how full I may be. 那怕是 廉價 不 好吃的東西 I promise not to waste water. 同時 我們 再也不要浪 費飲水 I pray that this little boy be alleviated from 大家來 為下面這 個兒童 祈禱 his suffering. 祈望他能 脫離苦海 I pray that we will be more sensitive towards the suffering 祈禱上 蒼 讓我 們能 多感受 到 in the world around us and 發生在 我們週 遭這 些苦難 訊息 not be blinded by our own selfish nature and interests. 能用良 知去體認 不要 刻意忽 視或裝作無 睹 I hope this picture will always serve as a reminder to us 下面這 張圖 希望能提 醒我們 about how fortunate we are and 大家的生 活是 多美好 that we must never ever take things for granted. 今天享 有的 一切 都是 應該 的 更不要 錯誤認 為 Think & look at this... when you complain about your food and the food we waste daily..." 以後 再也 不要抱 怨我 們食物 不好 或浪費 食物 MAY ALL HUMAN BEINGS BE FREE FROM SUFFERING!!!! 期望人類都能離苦 keep on forwarding it to all our friends. On this good day, let's make a prayer for the suffering in any place around the globe and Please don't break this, send this friendly reminder to others. 請幫忙 傳送 並祈求上 蒼減輕 他們 苦難 1994 年 普立茲 新聞獎 照片 PLEASE, MY GREAT FRIENDS, PLEASE, MY GREAT FRIENDS, DON'T BREAK THIS CHAIN, PLEASE, MY GREAT FRIENDS, DON'T BREAK THIS CHAIN, KINDLY SEND IT TO SOMEONE YOU LOVE, PLEASE, MY GREAT FRIENDS, DON'T BREAK THIS CHAIN, KINDLY SEND IT TO SOMEONE YOU LOVE, TO ENABLE HIM OR HER SEE WHAT GOD PLEASE, MY GREAT FRIENDS, DON'T BREAK THIS CHAIN, KINDLY SEND IT TO SOMEONE YOU LOVE, TO ENABLE HIM OR HER SEE WHAT NATURE HAS DONE IN HIS/HER LIFE COMPARED PLEASE, MY GREAT FRIENDS, DON'T BREAK THIS CHAIN, KINDLY SEND IT TO SOMEONE YOU LOVE, TO ENABLE HIM OR HER SEE WHAT NATURE HAS DONE IN HIS/HER LIFE COMPARED WITH THESE KIDS' DEPLORABLE PLEASE, MY GREAT FRIENDS, DON'T BREAK THIS CHAIN, KINDLY SEND IT TO SOMEONE YOU LOVE, TO ENABLE HIM OR HER SEE WHAT NATURE HAS DONE IN HIS/HER LIFE COMPARED WITH THESE KIDS' DEPLORABLE CONDITIONS. PLEASE, 希望這檔案 不要只傳到你手中就中斷
2008年8月23日土曜日
2008年8月18日月曜日
Visualization Examples
Read this document on Scribd: Visualization Examples
Visualization Methods Exploration Introduction Over at Visual-Literacy.org , Ralph Lengler and Martin Eppler have published this great Periodic Table of methods of visualisation. This displays around 100 diagram types, with examples and a multi-faceted classification by: q q q q q simple to complex data / information / concept / strategy / metaphor / compound process / structure detail / overview divergence / convergence In this application I have borrowed the list of diagrams to create an alternative interface, partly as an exericise in XML processing with XQuery and the eXist native XML database, on whose site this is currently hosted.. There is an entry in the DSA Module Blog which describes the process of scaping the file to get this data. Methods can be categorised in various ways. The Periodic table shows several categorisations in the same graphic, but there are other ways to categorise, and hence find a method.. To support an exercise with my students, I've created a very simple application for a viewer to group methods and give the group a tag. Later I will look at doing something with these groups, such as a tag cloud. The Application q q q q List all Methods List all the groups of methods Create a new group of methods Implementation Chris Wallace Diagrams tagged Compound Visualization by vizlit Home Edit Tag show/ hide sample cartoon Google Images Wikipedia show/ hide cognitive sample mapping Google Images Wikipedia show/ hide graphic sample facilitation Google Images Wikipedia show/ hide sample infomural Google Images Wikipedia show/ hide knowledge sample map Google Images Wikipedia show/ hide learning sample Google map Images Wikipedia rich picture show/ hide sample Google Images Wikipedia Diagrams tagged Concept Visualization by vizlit Home Edit Tag show/ hide sample argument slide Google Images Wikipedia cause effect chains show/ hide sample Google Images Wikipedia show/ hide communication sample diagram Google Images Wikipedia concentric circles show/ hide sample Google Images Wikipedia concept fan show/ hide sample Google Images Wikipedia show/ hide sample concept map Google Images Wikipedia concept skeleton show/ hide sample Google Images Wikipedia critical path method show/ hide sample Google Images Wikipedia show/ hide sample decision tree Google Images Wikipedia dilemma diagram show/ hide sample Google Images Wikipedia evocative knowledge show/ hide sample map Google Images Wikipedia flight plan show/ hide sample Google Images Wikipedia force field diagram show/ hide sample Google Images Wikipedia gantt chart show/ hide sample Google Images Wikipedia show/ hide ibis sample argumentation Google map Images Wikipedia layer chart show/ hide sample Google Images Wikipedia show/ hide sample meeting trace Google Images Wikipedia mindmap show/ hide sample Google Images Wikipedia minto pyramid technique show/ hide sample Google Images Wikipedia perspectives diagram show/ hide sample Google Images Wikipedia pert chart show/ hide sample Google Images Wikipedia show/ hide process event sample chains Google Images Wikipedia soft system modeling show/ hide sample Google Images Wikipedia square of oppositions show/ hide sample Google Images Wikipedia Wikipedia show/ hide information sample lens Google Images Wikipedia show/ hide parallel sample coordinates Google Images Wikipedia swim lane diagram show/ hide sample Google Images Wikipedia show/ hide sample synergy map Google Images Wikipedia show/ hide sample toulmin map Google Images Wikipedia show/ hide sample vee diagram Google Images Wikipedia Diagrams tagged Data Visualization by vizlit Home Edit Tag show/ hide sample area chart Google Images Wikipedia show/ hide sample bar chart Google Images Wikipedia show/ hide cartesian sample coordinates Google Images Wikipedia show/ hide sample continuum Google Images Wikipedia show/ hide sample histogram Google Images Wikipedia show/ hide sample line chart Google Images Wikipedia pie chart show/ hide sample Google Images Wikipedia show/ hide sample scatterplot Google Images Wikipedia show/ hide sample spectrogram Google Images Wikipedia table show/ hide sample Google Images Wikipedia tukey box plot show/ hide sample Google Images Wikipedia Diagrams tagged Information Visualization by vizlit Home Edit Tag show/ hide sample clustering Google Images Wikipedia show/ hide cone-tree sample diagram Google Images Wikipedia cycle diagram show/ hide sample Google Images Wikipedia show/ hide data flow sample diagram Google Images Wikipedia show/ hide sample data map Google Images Wikipedia show/ hide entity sample relationship Google diagram Images Wikipedia show/ hide sample flow chart Google Images Wikipedia show/ hide hyperbolic sample Google tree Images petri net show/ hide sample Google Images Wikipedia radar chart show/ hide sample Google Images Wikipedia sankey diagram show/ hide sample Google Images Wikipedia semantic network show/ hide sample Google Images Wikipedia timeline show/ hide sample Google Images Wikipedia treemap show/ hide sample Google Images Wikipedia venn diagram show/ hide sample Google Images Wikipedia Diagrams tagged Metaphor Visualization by vizlit Home Edit Tag bridge show/ hide sample Google Images Wikipedia funnel show/ hide sample Google Images Wikipedia show/ hide heaven n sample hell chart Google Images Wikipedia show/ hide iceberg sample diagram Google Images Wikipedia metro map show/ hide sample Google Images Wikipedia show/ hide parameter sample ruler Google Images Wikipedia show/ hide sample story template Google Images Wikipedia temple show/ hide sample Google Images Wikipedia tree show/ hide sample Google Images Wikipedia Diagrams tagged Process Visualization by vizlit Home Edit Tag show/ hide communication sample Google diagram Images Wikipedia critical path method show/ hide sample Google Images Wikipedia show/ hide sample cycle diagram Google Images Wikipedia data flow diagram show/ hide sample Google Images Wikipedia decision discovery diagram show/ hide sample Google Images Wikipedia show/ hide sample decision tree Google Images Wikipedia flight plan show/ hide sample Google Images Wikipedia flow chart show/ hide sample Google Images Wikipedia funnel show/ hide sample Google Images Wikipedia gantt chart show/ hide sample Google Images Wikipedia hype cycle show/ hide sample Google Images Wikipedia life cycle diagram show/ hide sample Google Images Wikipedia show/ hide sample meeting trace Google Images Wikipedia metro map show/ hide sample Google Images Wikipedia pert chart show/ hide sample Google Images Wikipedia petri net show/ hide sample Google Images Wikipedia show/ hide process event sample chains Google Images Wikipedia s-cycle show/ hide sample Google Images Wikipedia soft system modeling show/ hide sample Google Images Wikipedia swim lane diagram show/ hide sample Google Images Wikipedia system dynamics show/ hide sample Google Images Wikipedia technology roadmap show/ hide sample Google Images Wikipedia timeline show/ hide sample Google Images Wikipedia value chain show/ hide sample Google Images Wikipedia Diagrams tagged Strategy Vizualisation by vizlit Home Edit Tag affinity diagram show/ hide sample Google Images Wikipedia bcg matrix show/ hide sample Google Images Wikipedia decision discovery diagram show/ hide sample Google Images Wikipedia edgeworth box show/ hide sample Google Images Wikipedia show/ hide sample failure tree Google Images Wikipedia feedback diagram show/ hide sample Google Images Wikipedia house of quality show/ hide sample Google Images Wikipedia hype cycle show/ hide sample Google Images Wikipedia ishikawa diagram show/ hide sample Google Images Wikipedia life cycle diagram show/ hide sample Google Images Wikipedia magic quadrant show/ hide sample Google Images Wikipedia show/ hide mintzbergs sample organigraph Google Images Wikipedia show/ hide organisation sample Google chart Images Wikipedia show/ hide performance sample charting Google Images Wikipedia porters five forces show/ hide sample Google Images Wikipedia portfolio diagram show/ hide sample Google Images Wikipedia s-cycle show/ hide sample Google Images Wikipedia spray diagram show/ hide sample Google Images Wikipedia show/ hide stakeholder sample Google map Images Wikipedia show/ hide stakeholder sample rating map Google Images Wikipedia show/ hide sample strategic game board Google Images Wikipedia strategy canvas show/ hide sample Google Images Wikipedia show/ hide sample strategy map Google Images Wikipedia supply demand curve show/ hide sample Google Images Wikipedia taps show/ hide sample Google Images Wikipedia technology roadmap show/ hide sample Google Images Wikipedia show/ hide sample value chain Google Images Wikipedia show/ hide zwickys sample morphological Google box Images Wikipedia
Empirical Studies of Information Visualization
Read this document on Scribd: Empirical Studies of Information Visualization
Chapter 6 Empirical Studies of Information Visualization The farther back you can look, the farther forward you are likely to see. Winston Churchill The purpose of an empirical study is to discover and explain facts and factual relationships. The dictionary defines the term “empirical” as meaning “relying on or derived from observation or experiment”. In philosophy of science, inductivism refers to the belief that scientific knowledge is derived from the facts of experience acquired by observation and experiment, and that science is objective (Chalmers, 1982). This view first became popular due to the Scientific Revolution in the 17th century, especially the work of great pioneering scientists such as Galileo and Newton. The foundation of inductivism – the nature of observation – has been studied and challenged by philosophers over the past few hundred years. Chalmers describes the problem with inductivism as:“two normal observers viewing the same object from the same place under the same physical circumstances do not necessarily have identical visual experiences, even though the images on their respective retinas may be virtually identical.” In other words, our visual experience is influenced by our past experience, our knowledge, and our expectation. This in part explains why Galileo’s rivals could not see through Galileo’s telescope what Galileo could see, and why an X-ray expert could tell much more than a novice student from the same X-ray photograph of a patient. Modern philosophy of science has advanced much further beyond inductivism in understanding the nature of scientific knowledge and scientific methods. Interested readers are recommended to plunge into the vast literature ocean of philosophy of science. In this chapter, we will restrain ourselves and focus on empirical studies of information visualization. A study is counted as an empirical study if it explores and analyzes data associated with the use of information visualization functions as well as systems. Until recently, the number of empirical studies of information visualization techniques and applications has been rather small in comparison to the overall growth of the field, but the situation is changing. 6.1 Introduction The 1990s witnessed leaps and bounds in the field of information visualization with increasingly powerful techniques and visually appealing information visualization artifacts (Card et al., 1999; Chen, 1999a; Ware, 2000; Spence, 2001). Research in information visualization has been traditionally dominated by sophisticated and eyepopping innovations of visual representations and technical mechanisms. In contrast, empirical evaluations of information visualizations are often overshadowed by the enthusiasms for what can be done rather than for what should be done. 174 Information Visualization George Robertson, the inventor of the classic Cone Trees (Robertson et al., 1991), raised the issue in his keynote speech at the 1998 IEEE Information Visualization Symposium. Evaluations of information visualization techniques began to emerge, but the pace needs to be faster and the probe needs to be deeper. In 2000, I and Mary Czerwinski guest edited a nine-article special issue on empirical evaluations of information visualizations in International Journal of Human– Computer Studies, featuring two thematic streams: (1) evaluations of classic information visualization techniques and (2) applications of information visualization techniques in practical settings. Also in 2000, we guest edited another special issue in Journal of the American Society for Information Science on individual differences in virtual environments. For a fast-advancing field like information visualization, empirical evaluations are particularly important. Empirical evidence is an integral part of our domain knowledge, especially on what works, what failed, or what remains unknown. This is also a good opportunity for a fast-expanding field to incorporate valuable experiences and adapt established methodologies from such relevant disciplines as Human–Computer Interaction (HCI) and psychology. Indeed, a substantial proportion of empirical studies have appeared in HCI-related journals and conferences. Obviously, the home discipline of an application domain should always get involved. In this chapter, we introduce some of the most representative empirical studies of information visualization, not only to outline the proven knowledge of the domain, but also to stimulate more empirical evaluations of information visualizations. Every empirical study addresses issues concerning users, tasks, visualization designs, and the visualized information, although individual studies may differ in terms of their focus and approaches. The organization of this chapter essentially follows this sequence: 1 Users, including cognitive factors such as spatial ability, spatial memory, and associative memory, as well as the effects of gender and prior knowledge. 2 Tasks, including visual search tasks, visual scanning, detecting shortest paths, judging visualized quantitative information. 3 Visual Features, including focus context views, 2D versus 3D. 4 Visualized Information, including hyperlinks of a website and networks of documents. Figure 6.1 is a co-citation network derived from empirical studies of information visualization between 1993 and 2003. The visualization was generated by CiteSpace (see Chapter 8).A few pivot points in the network are worth noting. The Cone Trees paper by Robertson et al. (1991) is located in the center of the map, joining networks from four of the six time slices. Shneiderman’s 1992 tree maps paper is also a pivot point, connecting three slices since 1990. Chernoff ’s (1973) paper connects two slices. So do Inselberg and Dimsdale’s 1990 paper on parallel coordinates and LeBlanc, Ward, and Wittels 1990 paper. Tufte’s 1990 book connects three time slices between 1997 and 2002. Such images filtered by citation and co-citation strengths directly reflect where the focus of empirical studies has been. We begin with a meta-analysis of empirical studies of mainly visual information retrieval tasks, followed by more generic perceptual tasks. We attempt to maintain focus on such tasks as the starting point of each empirical study. Non-experimental studies are also introduced in this chapter. The intended message is that empirical studies should take a broader view. Empirical Studies of Information Visualization 175 Figure 6.1 A co-citation map of empirical studies in information visualization based on a sequence of two-year time slices (1993–2003). 6.2 Meta-analysis A number of fundamental issues must be addressed for the further development of the information visualization field. What is the central research question that most studies aim to address? What is the optimal task–feature taxonomy for information visualization design? What is the most commonly used experimental design? Is there any consensus that one can draw from the existing empirical findings in the literature? What is the most powerful visualization feature for a given task? To what extent are the current empirical findings consistent across different studies? The subsequent meta-analysis used the same methodology as the meta-analysis of hypertext systems (Chen and Rada, 1996). This meta-analysis focuses on the field of information visualization. A comprehensive report of the meta-analysis can be found in (Chen and Yu, 2000). As we will see, this is only the first step to building a task–feature taxonomy that can accommodate empirical evidence concerning various information visualization technologies. 6.2.1 Sampling Empirical Studies The meta-analysis focuses on experimental studies that include at least one of the three types of independent variables, users, tasks, and tools. User-related variables are mainly represented by individual differences in terms of cognitive factors; toolrelated variables include a variety of information visualization features. Measures to do with users include several cognitive factors such as associative memory (MA), spatial ability (VZ) and visual memory (MV). However, because of the small number of papers that directly address these cognitive factors, they are excluded from the meta-analysis. Instead, we will discuss some of these studies in the second half of this chapter. 176 Table 6.1 A classification of meta-analytical methods Diffuse Significance Level Compare Combine A E Effect Size B E Focused Information Visualization Significance Level C F Effect Size D F The meta-analysis is concerned with two broadly defined types of dependent variables: accuracy and efficiency. Accuracy measures typically include precision, error rate, the average number of incorrect answers, and the number of correct documents retrieved. Efficiency measures typically include the average time to completion, and the performance time. In terms of tools, interfaces that support visual–spatial features, such as cone trees, information landscape, associative networks and multidimensional scaling solutions, are valid candidates. Table 6.1 shows a classification of six meta-analytical methods. A meta-analysis is determined by three factors. A diffuse study formulates and tests a null hypothesis, whereas a focused study tests not only a hypothesis of significant difference, but also the direction of the difference.A focused study requires a better understanding of the underlying relationship to formulate the more specified hypotheses. Since a combining meta-analysis does not distinguish diffuse and focused studies, there are a total of six ways to conduct a meta-analysis: (A) comparing diffuse studies of significance testing, (B) comparing diffuse studies of effect size estimation, (C) comparing focused studies of significance testing, (D) comparing focused tests of effect size estimation, (E) combining studies of significance testing and (F) combining studies of effect size estimation. The aim of the meta-analysis of information visualization studies is to find invariant patterns in the existing body of empirical evidence. It focuses on the combined findings concerning the following hypotheses. • • H1 – Effects of users’ cognitive ability: – Users with stronger cognitive ability perform more accurately with information visualization systems than users with weaker cognitive ability. – Users with stronger cognitive ability perform more efficiently with information visualization systems than users with weaker cognitive ability. H2 – Effects of information visualization: – Users perform better, in terms of accuracy or efficiency, with interfaces with visualization components than with interfaces without such features. Studies were selected from the results of a comprehensive bibliographic search on published studies indexed by keywords such as visualization, image, graphics, evaluation, empirical and experiment. The following journals and conferences were searched in particular: • • • • • • Communications of ACM ACM Transactions on Computer–Human Interaction ACM Transactions on Information System ACM Transactions on Computer System ACM Transactions on Design Automation of Electronic System ACM Transactions on Database System Empirical Studies of Information Visualization 177 • • • • • • • • IEEE Computers Journal of American Society for Information Science IEEE Information Visualization Symposium (1995–1999) IEEE International Conference on Information Visualization. Each located study must meet the following selection criteria: The study must include an experiment design. The study must include at least one experimental condition in which a visual–spatial component appears in the user interface. The study must include at least one dependent variable on accuracy or efficiency. The study must report its results in sufficient detail, including F-test, t-test, correlation coefficients or p levels. The search located 35 evaluative studies published between 1991 and 2000. In particular, 32 studies (91%) were published between 1996 and 2000, indicating that the empirical evaluation of information visualization is still in its early stage. An array of information was coded for each study: independent variables, dependent variables, sample sizes, methods of assigning subjects, the background of the researchers, visual–spatial components used, the year of publication, tasks and statistics of significance tests. Among the 35 studies, eight studies were excluded in the first round. Seven studies were further removed because they only reported standard deviations and means. For studies that only report group means and standard deviations, the significance levels can be calculated as paired t-tests. However, it was decided that such studies should not be included. Eventually, six fully qualified studies entered the meta-analysis, investigating two broad types of causal relationships: 1 Effects of visual–spatial interfaces on information retrieval. 2 Effects of cognitive ability of users on information retrieval. Effects are measured by two categories of dependent variables: accuracy and efficiency. The magnitude of a specific relationship between two variables is often estimated by an effect size r, which can be calculated from a given one-tailed p value and the corresponding sample size. Tests of significance alone are not informative enough for practitioners and designers of visualization systems to judge the usefulness of a visualization feature. A meta-analysis typically compares and combines effect sizes and significance levels in the form of Fisher’s standard score zr and the standard normal deviate score Z. An effect size r is transformed to Fisher’s zr. For instance, an effect size r of 0.30 corresponds to Fisher’s zr of 0.31. Z scores can be obtained from reported onetailed p values of significance tests according to cumulative distribution functions, such as CDF and FCDF. Effect sizes in Fisher’s zr can be combined using standard formulae, which can be found in textbooks on meta-analysis (e.g. Rosenthal, 1987). The Z scores can be combined using Stouffer’s method (see Rosenthal, 1987). These two procedures of combination are recommended for their computational simplicity. Finally, the results of the combination need to be converted back to a correlation coefficient r as the combined effect size and a one-tailed p value as the combined significance level. For studies that only reported group means and standard deviations, the significance levels can be calculated as paired t-tests. However, it was decided to keep the 178 Information Visualization simple selection criteria and not to involve data that require additional processing at this stage. If results were reported as non-significant, a p value of 0.50 and a Z of 0.00 were coded. The heterogeneity test verifies whether a group of variables indeed represent the same underlying factor.A large heterogeneity implies a grouping that may not capture the variance of a group of results. According to Rosenthal (1987), the heterogeneity of a set of effect sizes refers to fluctuations from the average of the group. It follows a distribution of 2 with K 1 degrees of freedom, where K is the number of studies. The heterogeneity of significance levels has the same distribution. Chen and Yu (2000) grouped a number of accuracy measures, such as the average number of incorrect documents retrieved, and recall. The measures of efficiency such as the completion time, performance time, response time and search time, were lumped into another group. 6.2.2 Effects of Individual Differences on Accuracy Individual differences refer to users’ experience and their ability to use various visualization tools and their cognitive abilities in general. The synthesizing hypothesis states that users with stronger cognitive abilities, for instance, high VZ scores for spatial ability or high MA scores for associative memory, will benefit significantly more from visual–spatial interfaces than those with weaker cognitive abilities. The results from three studies were compared and combined. The combined effect size of cognitive abilities on accuracy is 0.60, which is usually regarded as a medium-to-large effect size. The combined significance level Z is 6.66 and this is statistically significant (p 0.001). Comparing the significance levels of diffuse studies yielded a statistically significant 2 ( 2 15.99, df 2, p 0.001). Comparing the effect sizes of diffuse studies was also found statistically significant ( 2 9.71, df 2, p 0.0078). These results have confirmed that the meta-analysis hypothesis, i.e. users’ cognitive abilities, have effects on accuracy with visualization interfaces. Comparing focused studies did not find a statistically significant linear trend associated with the degree of visual–spatial features in interfaces. Contrast weights were assigned to MDS, Aspect window, and NIRVE (globe) on comparing focused tests are (Zfocused-contrast-linear-test 0.9822, p 0.16) and (Zrfocused-contrast-linear-test 0.5, p 0.28). 6.2.3 Effects of Cognitive Abilities on Efficiency The meta-analysis hypothesizes that users with stronger cognitive abilities will perform more efficiently than users with weaker cognitive abilities. This hypothesis was supported by all the results from studies. Results supporting the hypothesis were assigned positive signs, and the negative signs indicate findings in the opposite direction. The combined effect size of users’ cognitive abilities is 0.59, which is statistically significant (p 0.005, one-tailed). The combined significance level Z is 6.66, which is also statistically significant (p 0.005, one-tailed). The significance levels in comparing diffuse studies are heterogeneous according to the heterogeneity test Empirical Studies of Information Visualization 179 ( 2 13.69, df 2, p 0.0002), which means that these findings are essentially different from each other. On the other hand, the heterogeneous test of effect sizes in diffuse studies is not statistically significant ( 2 3.9, df 2, p 0.14), which means, in terms of effect sizes, that these findings are similar to each other. There is a statistically significant linear tread in terms of effect size across this set of studies. Effect sizes on visual–spatial interfaces towards the high end tend to be smaller than those on visual–spatial interfaces towards the lower end (Zrfocused-contrast-linear-test 1.36, p 0.0043). These results suggest that given the same level of cognitive ability, users tend to perform better on less-sophisticated visualization interfaces. For example, users with MDS are likely to outperform their counterparts with NIRVE. 6.2.4 Effects of Visualization on Accuracy The meta-analysis considered five studies that tested the effects of visualization. The hypothesis was that users perform better with visual–spatial informationretrieval interfaces than traditional retrieval interfaces. This hypothesis was supported by four out of the five studies. The combined effect size is small (r 0.089) according to Cohen (1977), but this is not statistically significant (p 0.234). The individual effect sizes significantly differ from each other. The combined significance level (Z 2.11) is also not statistically significant (p 0.05). Statistically significant discrepancies were found among both significance levels and effect sizes ( 2 39.89, df 4, p 0.000 and 2 64.12, df 4, p 0.000, respectively). The results of linear trend tests in focused studies did not show a statistically significant linear trend across the range of visual–spatial interfaces (Zrfocused-contrast-linear-test 0.464, p 0.32, one-tailed). For example, users did not perform increasingly better from MDS to the Data Mountain. Major conclusions from the meta-analysis can be summarized as follows. • • • • Empirical studies of information visualization are still very diverse and it is difficult to apply meta-analysis methods. Individual differences, including a variety of cognitive abilities, should be investigated systematically in the future. Given the same level of cognitive abilities, users tend to perform better with simpler visual–spatial interfaces. The combined effect size of visualization is not statistically significant. A larger homogeneous sample of studies would be needed to expect conclusive results. This is the first attempt in raising the awareness that it is crucial to conduct empirical studies concerning information visualization systematically within a comparable reference framework. As the number of studies on similar visualizations increases, we expect that regularly conducted meta-analyses would be particularly useful to help us to improve our understanding of the empirical aspect of the field as a whole. The meta-analysis eliminated many studies because they failed the conventional selection criteria for a meta-analysis one way or the other. In order to improve the quality, clarity and comparability of experimental studies of information 180 Table 6.2 A list of empirical studies of information visualization Study Cleveland and McGill Hollands et al. Ware and Franck Stanney and Salvendy Sutcliffe and Patel Darken and Silbert Schaffer et al. Year 1984 1989 1994 1995 1996 1996 1997 Spatial ability Users Tasks Reading quantitative information Trip planning with subway networks Manipulating, moving or rotating, graphs Accessing a hierarchical information structure Navigating in virtual reality Path finding in hierarchically clustered graphs Searching documents Long term spatial memory of manually placed document icons Meta analysis Identifying nodes and edges that are necessary for keeping a graph connected Searching documents Information Visualization Visualizations Framed-rectangle charts Fisheye view 2D vs. 3D 2D vs. 3D Landmarks as navigational cues Fisheye view vs. full zoom views Pathfinder networks of documents Data Mountain planar surface with passive landmarks on landscape texture Meta analysis Graph drawing aesthetics Chen and Czerwinski Robertson et al. 1997 1998 Spatial ability Spatial memory Chen and Yu Purchase 2000 2000 Meta analysis Chen 2000 Spatial ability, associative memory, spatial memory Pathfinder networks of documents Risden et al. Stasko et al. 2000 2000 Westerman and Cribbin Cockburn and McKenzie Cockburn and McKenzie Cockburn and McKenzie Goguen Kobsa Chen et al. Czerwinski et al. Ware et al. Pirolli et al. 2000 2002 2001 Spatial memory Searching files/ directories Searching files/ directories visualized by space-filling algorithms Searching information nodes Hyperbolic 3D vs. hierarchical browser Treemap vs. Sunburst 2D vs. 3D MDS 2D vs. 3D physical vs. virtual 2D vs. 3D Data Mountain Cone Tree vs. standard tree browser 2000 2001 2001 2001 2002 2002 2003 Women Storing and retrieving web page thumbnail images Searching hierarchies of files and directories Pathfinder networks of documents Wide field of view with a large display Finding shortest paths Smoothness of a path Browsing a tree structure Hyperbolic view Searching documents Empirical Studies of Information Visualization 181 visualizations, future experimental studies of information visualizations should carefully take into account the following six aspects of an experimental design: 1 The use of standardized testing information. 2 The clarity of descriptions of visual–spatial properties of information visualizations. 3 The use of standardized task taxonomies for activities such as visual information retrieval, data exploration and data analysis. 4 The focus on the task–feature binding to be investigated in experimental studies. 5 The use of standardized cognitive ability tests. 6 The level of details in reporting statistical results. Some of the resources are available and some are yet to be developed to enable us to carry out experimental studies at a larger scale of consistency and comparability. For example, many experiments have already made use of the data collections prepared by NIST for the TREC Conference series. These collections include not only documents but also pre-defined queries and relevance judgments given by domain experts. Factor-Referenced Cognitive tests have been widely used to measure individuals’ cognitive abilities. Conventions of reporting statistical results should become a part of the standard instructions for authors in key journals and conferences in the field, for example, use p 0.078 rather than p 0.1. The more challenging issue is the design of realistic and practical tasks that can really put specific features of information visualization to the test. The provision of more task–feature taxonomies is certainly desirable so as to widen the range of our options in designing experimental studies. The development of task–feature taxonomies relies on a better understanding of how users make use of given visualization functions. To a large extent this is an adaptation process between users, available visualization functions and their tasks at hand; there is always more for us to find out. The above guidelines are recommended for reporting empirical findings in a more consistent and comparable manner. As a result, we will be able to utilize analysis and synthesis tools such as meta-analytical methods more effectively. We will be able to make sense of diverse and possibly conflicting empirical findings more confidently and systematically. Table 6.2 lists a collection of empirical studies. In the subsequent sections of this chapter, we will discuss some of the studies, although it is not feasible to cover all the studies in one place. 6.3 Preattentive and Elementary Tasks One way to categorize empirical studies is by the granularity of tasks performed by subjects.At the highest level, subjects may perform monolith and highly applicationoriented tasks using a dedicated computer system which has an information visualization component, for example, using a cone tree visualization to find PDF files that are related to IEEE Visualization 2003. In contrast, at a lower level, subjects may perform atomic and application-neutral tasks which may or may not directly involve an information visualization feature at the system level, for example, given a group of shapes on the screen, identifying the odd one out. Currently, there is still a huge gap between the two extremes in the literature of information visualization. One of the major challenges facing empirical studies of information visualization is the lack of a clear understanding of how people react upon various visual and spatial cues at the application level and how such cues are translated into 182 Information Visualization Figure 6.2 A filled circle can be preattentively detected in the left image, but not in the right one. observable actions. The best starting point is the lower level elementary tasks. Once we have a good understanding of how we perform elementary tasks, we may find ways to break down more complex tasks into elementary sub-tasks, or at least isolate the crucial element of a given task. 6.3.1 Preattentive Processing Human eye movements take about 200 milliseconds to initiate. Perceptual tasks that can be performed in less than this amount of time are called preattentive, because such tasks ought to be effortless for us to perform. This is about the amount of information that a single glimpse can pick up. A small number of visual properties are known to be preattentive and they can literally be spotted at a glance. Spotting an odd object from similar ones is a typical preattentive task. If the target object has a unique visual feature, then we can easily tell its presence or absence at a glance. In contrast, if the target object does not have a unique visual feature, then we have to perform a more detailed visual scan to determine whether the target is there or not. Searching for a filled circle in the left-hand side of Figure 6.2 is a preattentive task, whereas the same task is not preattentive to the right-hand side of Figure 6.2. Healey et al. (1996) collected a list of 2D visual features that can pop out during visual search. For example, the list includes studies by Juléz and Bergen (1983) on various elements in preattentive vision and perception of textures, and by Triesman (1985) on preattentive processing in vision. Table 6.3 is a list of preattentive visual features compiled by Healey et al. (1996). 6.3.2 Change Blindness Change blindness is a phenomenon in visual perception in which very large changes occurring in full view in a visual scene are not noticed (O’Regan, 2001). Change blindness is likely to occur if the changes are arranged to occur simultaneously with some kind of irrelevant, brief disruption in visual continuity, such as the large retinal disturbance produced by an eye saccade, a shift of the picture, a brief flicker, an eye blink, or a film cut in a motion picture sequence. These phenomena are attracting an increasing amount of attention from experimental psychologists and philosophers, because they suggest that humans’ internal representation of the visual world is much sparser than usually thought. Change blindness was first noticed in an experiment by McConkie and Currie (1996), which focused on the role of eye movements. In their experiment, observers viewed high-resolution, full-color everyday visual scenes on a computer monitor. Empirical Studies of Information Visualization Table 6.3 Preattentive visual features (Healey et al., 1996). Feature Line (blob) orientation Length Width Size Curvature Number Terminators Intersection Closure Color (hue) Intensity Flicker Direction of motion Binocular luster Stereoscopic depth 3D depth cues Lighting direction 183 Their eye movements were measured. The computer could make changes in the scene as a function of where the observer looked. For example, when the observer looked from the door of a house to the window, the window changed in some way: it could disappear, be replaced by a different element, change color, or change position. When the change occurred during an eye movement, surprisingly large changes could be made without being noticed by observers. Elements of the picture that occupied as much as a fifth of the picture area would not be seen. At first, the phenomenon was assumed to have something to do with the mechanisms the brain uses to combine information from successive eye fixations to form a unified view of the visual world. In particular, every time the eye moves, the retinal image shifts. Some mechanism in the brain may correct for such shifts in order to create a stable view of the world. However the mechanism could be imperfect and not take into account certain differences in the visual content across the shift, thereby explaining why changes made during saccades might sometimes go unnoticed. Further research has revealed that the change blindness phenomenon was not specifically related to eye movements. Instead of focusing on an eye movement, Rensink, O’Regan, and Clark (1997) introduced a brief flicker between successive images. For example, one picture was first shown for 250 ms, followed by a brief blank screen for about 80 ms before a modified picture was shown. The 80 ms is about the same duration of an eye movement. Observers were told that something was changing in the picture every time the flicker occurred, and they were asked to search for it. Under conditions where no flicker was inserted in between the pictures (A-B-A-B-…) the change was immediately visible and totally obvious (Figure 6.3). However, with the flicker it was often much more difficult to locate the change, especially for changes that were not of “central interest” in the scene. For example, 184 Information Visualization Figure 6.3 Can you see the difference between the two pictures? (O’Regan, 2001). Reprinted with permission of J.K. O’Regan. the reflection of houses in a lake scene, though occupying a very large part of the picture, would not be considered to be what the picture was about. Observers sometimes were unable to see such changes at all, even after searching actively for as long as one minute. On the other hand, the changes were perfectly visible once they were pointed out to observers.1 The flicker technique shows that change blindness could occur without synchronizing the change with eye movements. This observation led to the re-examination of earlier experiments where changes were synchronized with eye movements. Once it was realized that change blindness was not specifically related to eye movements, but to the brief disruption that is inserted between the two versions of the picture, a large number of follow-up experiments were performed, mainly concerning three categories of disruptions: global disruptions, local disruptions, and progressive changes (Simons and Levin, 1997). Global disruptions are the ones that cover the whole area of the picture. The flicker experiments are global disruption experiments, since the blank displayed briefly between original and modified images covers the whole picture. Other global disruption examples are eye blinks, picture shifts, and film cuts.An additional, amusing, variant of the experiments with global disruptions are experiments in which the change occurs in real life. In a typical scenario described by Simons and Levin (1998), the experimenter stops a person in the street and asks for directions. While the person is speaking to the experimenter, workers carrying a door pass between the experimenter and the person, and an accomplice takes the place of the experimenter. The person usually goes on giving directions after the interruption, and very often does not notice that the experimenter has been replaced by a different person. Local disruptions are limited to five or six small, localized disturbances superimposed on the picture, just like mud splashes on a car windscreen (O’Regan et al., 1999). The disturbances can be small in comparison to the size of the change and they may have nothing to do with the location of the change. Slow changes involve no local or global disruption at all. Slow changes are hard to detect.2 Nowell et al. (2001) illustrated the change blindness problem with information visualization in SPIRE when users were using the Time Slicer. Users found 1 2 http://nivea.psycho.univ-paris5.fr/ECS/kayakflick.gif http://nivea.psycho.univ-paris5.fr/ECS/sol_Mil_cinepack.avi Empirical Studies of Information Visualization 185 Figure 6.4 Changes from one time slice to another were aided with wireframes and variable translucency (Nowell et al., 2001). The figure was produced at the Pacific Northwest National Laboratory, which is managed and operated by the Battelle Memorial Institute on behalf of the United States Department of Energy. © 2001 IEEE. Reprinted with permission. themselves unable to identify what had changed from one time period to the next. And they were unable to remember what was different in the previous slice. Nowell et al. used a white wire-frame for the emerging contours to highlight the change. Simultaneously, the opacity and color saturation of the vanishing contours was reduced. The emerging contours gradually changed their translucent color to become brighter and less translucent (Figure 6.4). 6.3.3 Elementary Perceptual Tasks Good empirical studies are built on solid theoretical foundations. The absence of good empirical studies in a field could be the sign that the field lacks theoretical foundations. As early as 1975, Kruskal (1975) identified the lack of theoretical foundations in the study of graphical methods: “in choosing, constructing, and comparing graphical methods we have little to go on but intuition, rule of thumb, and a kind of master-to-apprentice passing along of information … There is neither theory nor systematic body of experiment as a guide.” Until recently, this would not have been a bad description of the empirical frontiers of today’s information visualization field. An intriguing exemplar of a good empirical study is due to Cleveland and McGill (1984). Not only is its very methodology applicable to studying today’s information visualization artifacts, but it also demonstrates how often we are trapped by ideas that are too familiar to us. Cleveland and McGill had a simple goal of finding out how accurately people perform a number of basic tasks for extracting quantitative information from graphs (Cleveland and McGill, 1984). For example, which task is simpler and more accurate: reading time from a digital clock display or reading time from an analog clock display? Similarly, we can compare the accuracy of such tasks as comparing positions on a common scale and comparing angles or color saturations. Cleveland and McGill illustrated the profound difference between reading from a bar chart and reading a pie chart. In a bar chart, judging position along a common scale is the primary task, whereas the primary task in a pie chart is judging angles, arc lengths, and areas of pie slices. It may be intuitive to us that the bar chart reading task is probably more accurate than the pie chart reading task, but how do we find out for sure and how do we extend such comparisons to a wider range of information visualization designs? Cleveland and McGill’s study provides an excellent example. Cleveland and McGill argued that once an elementary perceptual task is empirically proven to be more accurate than others, it becomes possible to design a 186 Information Visualization graphical representation such that people will interact with the graphical representation through more accurate tasks. The Cleveland–McGill study ranks ten elementary perceptual tasks as follows, 1 being the most accurate, and 6 the least: 1 2 3 4 5 6 Positions along a common scale. Positions along non-aligned scales. Length, direction, angle. Area. Volume, curvature. Shading, color saturation. What is the basis of this ranking? In fact, they used power laws of theoretical psychophysics (Stevens, 1975) – if p is the perceived magnitude and a is the actual magnitude, then p is related to a by p ka . If a1 and a2 are two such magnitudes and p1 and p2 are corresponding perceived values, then p1/p2 (a1/a2) . The value of is therefore an indicator of the accuracy of the perception. When is 1, the perceived scale is the same as the actual physical scale. The closer is to 1, the more accurate the perceived value. Baird (1970) reviewed a large number of experiments in an attempt to determine the value of . It was discovered that values of tend to be reasonably close to 1 for length judgments, smaller than 1 for area judgments, and even smaller for volume judgments. This means that people tend to judge lengths accurately, underestimate areas, and even more underestimate volumes. Cleveland and McGill tested their ranking system in two experiments. They asked subjects to estimate the percentage that one value represents of a larger value. Such values were depicted as lengths, positions, and angles. Their experiments found that position judgments were the most accurately performed, followed by length judgments and angle judgments. More specifically, position judgments were between 1.4 and 2.5 times as accurate as length judgments, and 1.96 times as accurate as angle judgments. The main message of their study is: graphs should engage users in elementary tasks as high in the accuracy ranking as possible. They demonstrated how this principle can be applied to bar charts, pie charts, and statistical maps with shading. They recommended dot charts, dot charts with grouping, and framed-rectangle charts as alternative representations. In fact, framed-rectangle charts are one level higher in the accuracy ranking hierarchy, and they lead to more accurate estimates. They went into further details to demonstrate how a framed-rectangle chart solves a serious problem of shade- or color-coded statistical maps, such as the 1978 murder rate map of Gale and Halperin (1982). The primary elementary task for understanding Gale and Halperin’s murder rate map is mainly shading perception and comparison (Figure 6.5). The accuracy of shading-based judgments is at the bottom of the Cleveland–McGill accuracy ranking hierarchy. The goal was to transform the lower ranking task to a higher ranking task. Framed-rectangle charts are a better alternative because they require perceptual tasks based on judging position along a common scale, which can be performed more reliably (Figure 6.6). Cleveland and McGill cautioned that the accuracy criterion should be taken into account in a broader context – the power of a graphical representation is its ability to reveal patterns and structures not readily shown by other means. One of the most promising approaches to studying elementary perceptual tasks is eye tracking. Although eye tracking itself is by no means easy enough to be integrated into every empirical study of information visualization, it has the potential to identify the source of attraction from our information visualization design Empirical Studies of Information Visualization 187 Figure 6.5 The shade-coded murder rate map (Gale and Halperin, 1982). Reprinted with permission from the American Statistician. © 1982 by the American Statistical Association. All rights reserved. Figure 6.6 A framed-rectangle rework of the murder rate map (Cleveland and McGill, 1984). Reprinted with permission from the Journal of the American Statistical Association. © 1984 by the American Statistical Association. All rights reserved. and track ultimately what types of visual attributes attract the most attention. Figure 6.7 shows an example of what the eye fixation-worn photograph can tell us about where viewers were paying attention. 6.3.4 Semiotics and Semiology Moving from elementary perceptual tasks to application-oriented tasks requires a careful mapping between visual representations of information and how they may be interpreted by human users. Research in semiotics is one of the most relevant areas, although much has to be done to incorporate semiotics into empirical studies of information visualization. 188 Information Visualization Figure 6.7 Titanic. © 2003 Tobii Technology (www.tobii.se). Reprinted with permission of Henrik Eskilsson. Semiotics and semiology are closely related to each other but distinct. Semiotics is essentially due to the American philosopher, mathematician, and logician Charles Saunders Peirce, whereas semiology is primarily developed by the Swiss linguist Ferdinand de Saussure. What distinguishes the two schools of thought is the concept of interpretant, which is from Peirce’s model. In de Saussure’s linguistically oriented semiology, a sign – a word – is fixed by the text in which the word occurs. In semiology, two things are involved with the sign: the signifier and the signified. This approach is particularly suitable for a linguistic sign, but becomes problematic for a visual sign. Unlike a linguistic sign, a visual sign does not usually have an immediate and explicit context; therefore, a visual sign may appear as context independent. In other words, a visual sign is subject itself much more to changes in its implicit context. Peirce added the concept of interpretant to resolve the problem. Peirce’s work is regarded as the best source to develop a design or visual communication theory on semiotics. Codes are not only one of the fundamental concepts in semiotics, but also a sense-making framework in which signs become meaningful. The framework serves as interpretative devices. Scholars in semiotics distinguish social codes, textual codes and interpretative codes. Within a code there may also be “subcodes”: such as stylistic and personal subcodes (or idiolects). Some codes are fairly explicit; others are much looser, called “hermeneutics” by Guiraud. Semiotics believes that a sign does not have an “absolute” value in itself; rather, its value is dependent on its relations with other signs within the signifying system as a whole. When looking at a cave painting, or the Pioneer’s plaque, how is the viewer supposed to understand and interpret what they see? To be able to understand the message, the viewer needs to have prior knowledge of conventions on what a sign means and what a signifier connects to a particular domain. Information visualization is particularly in need of a generic theory that can help designers and analysts to assess information visualization designs. Goguen (2000) Empirical Studies of Information Visualization 189 developed a computational theory called semiotic morphisms on preserving the meaning of signs in translating symbol systems. He demonstrated the potential of semiotic morphisms in identifying defects of information visualization. From the semiotic mapping point of view, fundamental issues in information visualization can be understood in terms of representation: a visualization is a representation of some aspects of the underlying information; and the main questions are what to represent, and how to represent it. Information visualization needs a theory of representation that can take account not just of the capabilities of current display technology, but also of the structure of complex information, such as scientific data, the capabilities and limitations of human perception and cognition, and the social context of work. However, classical semiotics, which studies the meaningful use of signs, is not good enough, because it has unfortunately not been developed in a sufficiently rigorous way for our needs, nor has it explicitly addressed representation; also, its approach to meaning has been naive in some crucial respects, especially in neglecting (though not entirely ignoring) the social basis and context of meaning. This is why semiotics has mainly been used in the humanities, where scholars can compensate for these weaknesses, rather than in engineering design, where descriptions need to be much more explicit.Another deficiency of classical semiotics is its inability to address dynamic signs and their representations, as is necessary for displays that involve change, instead of presenting a fixed static structure, e.g. for standard interactive features like buttons and fill-in forms, as well as for more complex situations like animations and virtual worlds. Goguen’s initial applications of semiotic morphisms on information visualization have led to several principles that may be useful in assessing a range of information visualization design (Goguen, 2000). He suggested three rules of thumb: • • • Measuring quality by what is preserved and how it is preserved. It is more important to preserve structure than content when a trade-off is forced. The need to take account of social aspects in user interface design. The semiotics morphisms methodology is not just algebraic but also social. More in-depth studies are needed to verify the power of this approach. 6.4 Interacting with Trees Browsing a tree structure and finding an item in a hierarchy are the tasks by far the most extensively supported by the arsenal of information visualization. The most widely known information visualizations include cone trees, hyperbolic views, tree maps, and fisheye views. See Chapter 4 for a detailed description of each individual visualization. Since they support similar tasks, a typical empirical study in this group compares a visualization interface with the standard tree browser, namely the File Explorer in Windows on a personal computer. 6.4.1 Cone Trees Cockburn and McKenzie (2000) compared cone trees and an Explorer-like browser for exploring hierarchical data structures. In their test, the branching factors of cone 190 Information Visualization trees were limited to 20 or less in order to avoid cluttered cone trees. The original upper limit of a branching factor of 30 was noted in Robertson et al. (1991). They compared the performance of users browsing documents through a 3D cone tree interface and a normal tree browser. Browsing tasks were further divided into shallow browsing and deep browsing. Subjects in their experiment followed the paths more quickly with the normal tree browser than the 3D cone tree browser. Not surprisingly, it took longer to complete a deep browsing than a shallow browsing. As expected, it also took longer to follow paths in densely populated data structure. Despite the less efficient performance with the 3D interface, Cockburn and McKenzie noted possible reasons in subjects’ reports. For example, the 3D interface made it easier to see the structure and the search space. Subjects also felt that their lack of familiarity with the 3D cone tree interface affected their completion times, and they believed they would be able to become much more efficient with more experience. The relatively lower fidelity of the cone tree interface was also noted as a possible confounding factor. This study illustrates a challenging issue that empirical studies of innovative information visualization must address. Prior knowledge and practical experience are among the most predominating compounding factors. Longitudinal studies may resolve some of the problems. The challenging issue is also related to the granularity of the tasks studied. Browsing hierarchically organized files is a task that is so familiar to us, it becomes difficult to break away from the routine and think of how the work can be done differently. This is a general problem with the empirical study of information visualization. The 1984 Cleveland–McGrill study brilliantly demonstrates a potentially generic approach that can be adapted for the empirical study of today’s information visualization. 6.4.2 Tree Maps The TreeMap visualization technique is another classic technique that supports browsing hierarchically organized files on computer (Johnson and Shneiderman, 1991; Shneiderman, 1992). TreeMap utilizes a space-filling algorithm that fills recursively divided rectangle areas with components of a hierarchy. One of the most remarkable facts about tree maps is their continuous growth since the original design. It is both intriguing and informative to look at the evolution of tree maps in a broader context. Researchers at the Human–Computer Interaction Lab (HCIL) at the University of Maryland have done enormous innovative and stimulating work in information visualization, notably dynamic queries, tree maps, and zoomable user interfaces. The Maryland way to information visualization has several unique characteristics. Their vibrant approaches have been driven by a strongly user-centered philosophy, from direct manipulation, dynamic queries, to Shneiderman’s information visualization mantra. Important design decisions are made based on what users need. Another unique character of HCIL’s approaches is their emphasis on rigorous scientific methods. Every technical innovation comes with a series of persistent empirical evaluations and continuous refinement studies. Since the original tree map design in the early 1990s, the use, evaluation, and refinement of tree maps have led to a continuous stream of systems such as ordered tree maps and quantum tree maps. Research in zoomable user interfaces at HCIL has led to widely used toolkits such as Jazz and Piccolo. Empirical Studies of Information Visualization 191 The Craft of Information Visualization, written and edited by Bederson and Shneiderman (2003), features 38 seminal publications from their lab over the last two decades in information visualization. More importantly, it sets these seminal contributions to information visualization in the broader context of the field as a whole. Instead of attempting a partial coverage of the influential topics in this book, the reader is recommended to read The Craft of Information Visualization thoroughly. In this section, we start with an empirical study of TreeMap and SunBurst visualizations, followed by an example from HCIL on improving tree maps, namely ordered tree maps and quantum tree maps. We end this section with an empirical study of SpaceTree, also from HCIL. Although SpaceTree per se does not belong to the tree map family, its empirical evaluation reflects valuable insights into the visualization of hierarchical data structures. Stasko et al. (2000) compared TreeMap and SunBurst visualizations of hierarchies of computer files and directories. Both TreeMap and SunBurst can be regarded as space-filling visualizations. TreeMap fills a rectangle area completely (Figure 6.8), whereas SunBurst fills a circular area partially (Figure 6.9). In SunBurst, the top of the hierarchy is at the center of the visualization, whereas deeper levels of the hierarchy fan out away from the center and sub-directories are confined by their parent directories’ radial boundaries. The effects of the two visualization browsers on time to complete browsing tasks were tested on two hierarchies Figure 6.8 TreeMap visualization (Stasko et al., 2000). Reprinted with permission of John Stasko. 192 Information Visualization Figure 6.9 SunBurst visualization of a file directory’s structure. Reprinted with permission of John Stasko. of about 500 nodes each in the first experiment and about 3000 nodes each in the second experiment. The second independent variable phase was the order of two 16-task blocks. When tested with the smaller 500-node hierarchy in the first experiment, a main effect of browsers on task correctness was found, but the phase has no main effect and no interaction was found either. When tested with the larger 3000 node hierarchy in the second experiment, on the other hand, no main effect of browsers was found. Bederson et al. (2001) described the design of ordered and quantum treemaps in attempts to resolve some of the problems identified with earlier treemap designs. Rectangles are the building blocks of a treemap. Very thin rectangles tend to cause users problems. In general, a rectangle with a shape closer to square is easier to see from the user’s point of view. Therefore, balancing the aspect ratios of rectangles in a treemap has been proposed so as to avoid ultra-thin rectangles in a treemap. However, this solution introduces a new problem: it may no longer preserve the order of the underlying data and its stability is not guaranteed. The intrinsic order of data items is often an important clue in identifying visual patterns; for example, preserving the chronological order is an important requirement for an analysis of maturity and interest rate of bonds. In order to resolve this dilemma, the ordered Empirical Studies of Information Visualization 193 treemap algorithms, including strip and pivot algorithms, provide a trade-off solution. Ordered treemaps maintain the original order to an extent in treemaps while minimizing the presence of ultra-thin rectangles. As a result, ordered treemaps are characterized by layouts of partially ordered and close to square rectangles. Bederson et al. (2001) reported an interesting user study of ordered treemaps. They compared how long it took users to find specific rectangles among 100 rectangles mapped by different treemap algorithms, including squarified, pivot, and strip treemap algorithms. In terms of completion time measures, the expectation was that the fastest would be layouts created by the strip algorithm, then the pivot one, and finally the squarified algorithm. Although the results did not identify a clear winner between the strip and pivot maps, there was an indication of a trend consistent with the expectation: the strip was the fastest, followed by the pivot, which was in turn followed by the squarified. They went on to develop another improvement of treemaps called quantum treemaps. The reader is referred to The Craft of Information Visualization for detailed descriptions. As concrete exemplars, such a series of development, evaluation, and refinement efforts has practical implications on empirical evaluations of information visualization. It clearly shows the integral role of empirical evaluation and how it may become an indispensable component of a development methodology. We end this section with another example from HCIL. Plaisant et al. (2002) reflect on the evolution of the design of SpaceTree and a controlled experiment in which SpaceTree was compared with Microsoft Explorer and the hyperbolic view visualizations. Empirical evaluations of hyperbolic views are discussed in the next section. SpaceTree is not a space-filling algorithm like treemaps, but it addresses another type of space-filling questions. SpaceTree was directly motivated by the feedback from users on semantic zooming versus geometric scaling. One question is to do with geometric scaling when some of the fonts become too small to be readable: to show or not to show visible but unreadable texts? Instead of showing the unreadables, SpaceTree truncates unreadable levels of a hierarchy into expansible icons. In a way, this is in the same spirit of the way hyperbolic views treat items in surrounding areas. Earlier user studies of hyperbolic views found that a radical change of spatial configuration in hyperbolic geometry can cause disorientation problems. The need to re-focus when each time the display is updated is not a trivial cognitive burden. SpaceTree’s design particularly aims to maintain a stable and consistent layout. A controlled experiment was undertaken with 18 subjects working with a tree of more than 7,000 nodes visualized by Microsoft Explorer, Hyperbolic views, and SpaceTree, respectively. The test tasks were node search (e.g. find kangaroo), return to previously visited nodes (e.g. go back to kangaroo), and questions concerning the topological structure of the tree (e.g. given a branch find three nodes with more than 10 children nodes). The results indicate a substantial role of individual differences in terms of individuals’ performance and their preferences. For example, subjects found the Hyperbolic Browser was more “cool”than Explorer and SpaceTree,but preferred to use SpaceTree. Similar findings have been reported in the literature – people may be more impressed by a 3D interface, but prefer to use a 2D version nevertheless. Exactly what does it take to make something cool also something usable and vice versa? This is a generally recognized issue, but there is no readily convincing answer. Empirical studies at finer granularity are certainly necessary. In terms of performance measures, Explorer was the fastest in the first kangaroofinding tasks. Learning was expected to be a factor as SpaceTree became the fastest in the third task. Explorer was the winner in the returning-to-kangaroo tasks, 194 Information Visualization followed by SpaceTree; the Hyperbolic Browser took the longest time. SpaceTree was significantly faster than Explorer on topological tasks; the Hyperbolic Browser was in between. Examples in this section and indeed examples in this chapter highlight the challenging nature of empirical evaluation of information visualization. It would be much informative if a performance difference can be traced back to the original design decision and a problem can be resolved at the early stage. The crucial role of a user-centered philosophy is evident in the two examples from HCIL, which also confirms how much one can learn from the field of human–computer interaction along the way of empirical evaluation of information visualization. Users perform better with a stable layout, items in preserved orders, shapes that are easily readable, and many more. As one of the most widely tried visualizations, treemaps have the valuable opportunity to gather feedback from a wide range of users. The open source policy and the wide availability of treemap algorithms have also contributed to their proliferation and healthy improvements. 6.4.3 Hyperbolic Views Two empirical studies are highlighted below for the widely known focus context views. Risden et al. (2000) compared two conventional 2D hierarchical browsers and a browser with an interactive 3D hyperbolic view. They demonstrated where focus context views might become particularly useful for experienced users. Their results suggested that 3D interactive techniques might best be introduced alongside more familiar 2D visualizations so that the user can integrate interaction strategies. Empirical evidence so far appears to suggest a neck-and-neck situation in the 2D versus 3D studies, if not a winning 2D. Some of the empirical studies are discussed in earlier sections of this chapter. On the other hand, it should be noted that one should carefully examine the existing evidence and bear in mind the large amount of known and unknown confounding factors. One possibility is that file management tasks with 2D and 3D may require different perceptual and cognitive processes altogether at this level; therefore, it may be necessary to investigate tasks of much finer granularity. Some of us strongly believe that 3D visualizations should ultimately outperform their 2D counterparts. This is indeed a challenging and complex question, and it may never have a clear cut answer. Nevertheless, whenever there is a negative result to 3D, we are tempted to explain away the finding with confounding factors. The complexity of the issue is clear in a number of empirical studies associated with the evaluation of hyperbolic view browsers. Neither Lamping et al. (1995) nor Czerwinski and Larson (1997) find significant task performance improvements with a hyperbolic tree browser. However, when competing with other browsers in two competitions, the hyperbolic tree browser was the winner twice. Such mismatches between laboratory studies and field tests motivated a recent study by Pirolli et al. (2003). They studied the effects of information scent on visual search with a hyperbolic tree browser. Pirolli et al. (2003) introduced the concept of information scent into the race. Information scent is task-relevant display cues, such as node labels on a tree structure. At the beginning of this chapter we distinguish two types of tasks: visual– spatial tasks and semantic tasks, which are further refined to Type I and Type II tasks when dealing with graphs. Information scent can be seen as the symbolic reinforcements of spatial references. As a result, we can expect that such symbolic Empirical Studies of Information Visualization 195 reinforcements will make Type II tasks easy to perform. Therefore, if one performs Type II tasks with an interface, then information scent should make a difference. In this context, what Pirolli and his colleagues did becomes apparent. Information scent was quantified as an empirical accuracy of scent (AOS) score. Such AOS scores were used to characterize tasks to be performed. Task performance with different AOS scores was compared across a hyperbolic tree browser and the Microsoft Windows File Explorer. Two experiments were conducted. Although the first experiment found no overall difference in task completion time between the two browsers, there was an interesting finding: subjects performed better with the hyperbolic browser on high AOS tasks, whereas subjects performed better with the Windows File Explorer on low AOS tasks. Does it suggest that low AOS tasks are equivalent to Type I tasks? In other words, does it mean the Windows File Explorer works best with hierarchical organizations of straightforward symbolic references? The second experiment studied retrieval tasks only, and the analysis was enhanced by eye-fixation data. Hyperbolic tree users examined more nodes and visually searched through the tree hierarchy faster than users of an interface similar to the Windows File Explorer. Two factors affected visual search in the hyperbolic display: strong information scent improves visual search, and the density of a target region hinders visual search especially when information scent is weak. Earlier in this chapter, the Cockburn–McKenzie study found that an increased density in 3D cone trees makes tasks more difficult. Now similar results were found in 3D hyperbolic trees. There is at least one method that can definitely reduce the local density of a tree display, no matter in a Euclidean space or a hyperbolic space – using a fisheye view. 6.4.4 Fisheye Views Schaffer et al. (1997) compared fisheye views to traditional zooming on a hierarchically clustered network. Subjects were first asked to find a broken telephone line in a hierarchically clustered network and repair the broken phone line. Broken phone lines were displayed visually as red lines. The interface has a main effect on times for completing each task. People performed the task much faster when using fisheye views. In addition, subjects used zooms much less frequently with the fisheye view interface than with the full zoom interface. The significantly fewer number of zooming requests suggests that the problems associated with the density of tree visualization could indeed be resolved by a fisheye view. Hollands et al. (1989) compared task performance using fisheye and scrolling views. Subjects were asked to plan a trip with a subway map. Tasks involved locating stations, selecting optimal routes, and constructing optimal itinerary between stations on a subway network. The results slightly favored fisheye views. One of the problems with the fisheye viewer used in the study was that it moves the focal point to the center of display in an abrupt motion that caused disorientation. Carpendale et al. (1997) particularly addressed such disorientation problems. Users often experience disorientation when viewing distorted displays, such as fisheye views and hyperbolic views. Carpendale and her colleagues proposed to use visual cues such as shading and grid lines to help users correctly interpret the distortions. However, so far we are not aware of any comprehensive empirical studies that compare fisheye views with other types of information visualizations. 196 Information Visualization It is clear that more empirical evaluations of information visualizations are needed to address issues regarding the reliability and efficiency of individual visualization features. On the other hand, alternative research methods should be encouraged. 6.5 Interacting with Graphs One of the single most important ingredients for an empirical study is tasks. Tasks are not only an integral part of an empirical investigation, but also the key to understanding an empirical study as a whole.At the highest level, all interactions between users and information visualization can be characterized by the level of information that users act upon. Sometimes recognizing visual patterns may be all it takes to understand the visualized information. Sometimes, however, one has to dig deeper and think harder to make a connection and interpret what a visual representation means. We distinguish tasks involved in such situations by calling the former visual–spatial tasks, and the latter semantic tasks. Although in information visualization, tasks often go beyond the visual–spatial level, semantic tasks do not necessarily follow a visual–spatial task. Here we define two more specific types of tasks concerning graphs. One may perform two types of tasks when dealing with a graph, Type I and Type II. Type I tasks are purely concerned with topological properties of the graph, whereas Type II tasks are concerned not only with the topology, but also with the connections to the underlying phenomenon from which the graph is abstracted. Some typical examples of Type I tasks are: is the graph connected? Or, is node ni reachable from node nj? In contrast, examples of Type II tasks could be: what is Arnold Schwarzenegger’s Kevin Bacon number in the Hollywood movie stars network? Or, what rewiring can be done to make the North American electrical grids more resilient to a massive blackout? 6.5.1 Aesthetics and Legibility Until recently graph drawing algorithms have been evaluated exclusively in terms of their computational efficiency. Through a series of empirical studies, Helen Purchase, now at the University of Glasgow, Scotland, is pursuing a unique route to investigate the effects of various graph drawing aesthetics on human performance from a user-centered perspective (Purchase, 2000, 2002). Purchase primarily investigated Type I tasks in her studies. And she called these tasks relational, as opposed to interpretative Type II tasks. Purchase (2000) reported two experiments to study the effects of various graph drawing aesthetics on the error rates and the time to completion of subjects. In the first experiment, hand-drawn graphs were used with carefully varied graph drawing aesthetics as the independent variables. In the second experiment, graphs were drawn by algorithms. Five common graph drawing aesthetics were used in the first experiment (Purchase, 2000): • • minimize bends – minimize the total number of bends in polyline edges (Tamassia, 1987); minimize edge crossings – minimize the number of edge crossings in the display (Reingold and Tilford, 1981); Empirical Studies of Information Visualization 197 • • • maximize minimum angles – maximize the minimum angle between edges extending from a node (Coleman and Parker, 1996; Gutwenger and Mutzel, 1998); orthogonality – fix nodes and edges to an orthogonal grid (Tamassia, 1987; Papakostas and Tollis, 2000); symmetry – display a symmetrical view of the graph if possible (Gansner and North, 1998). Purchase tested the five primary hypotheses associated with these aesthetics. In addition, Tukey’s WSD pairwise comparison procedure (Gottsdanker, 1978) was used to determine if there were significant understandability priorities between the aesthetics. She found a significant effect of bends on errors, but not on response time; edge-crossing on both measures; angles on none; orthogonality on none; and symmetry on response time, but not on errors. More specifically, her results suggest that edge-crossing becomes particularly problematic when there are a large number of crossed edges. Her second experiment compared the error rates and response times on graphs drawn by eight graph layout algorithms, including Fruchterman and Reingold’s force-directed placement algorithm (Fruchterman and Reingold, 1991), and Kamada and Kawai’s layout algorithm (Kamada and Kawai, 1989). The main effect of the algorithms was statistically significant for errors, although not significant for response time. Kamada and Kawai’s algorithm has the lowest error rates. It also has the second lowest response time, but the difference is not statistically significant. Purchase (2002) compared 11 algorithms with respect to their measured presence of seven aesthetics, according to metric formulae, rather than based on human judgments. The 11 algorithms include spring-based algorithms and grid-based algorithms. All algorithms implicitly minimize the number of edge crosses. Springbased drawings are straight-line drawings, and they also implicitly favor symmetric layout structures. Grid-based drawings perform better than spring algorithms on the symmetry aesthetic. 6.5.2 Continuity as an Extrinsic Criterion The validity of graph drawing aesthetics are often taken for granted, rather than being derived from empirical evidence and theoretical foundations. Purchase (2000) identified edge crossings as the most important aesthetic based on two hand-drawn graphs, one with many crossings and one with few. Why are edge crossings bad? Are there more profound reasons behind the edge-crossing problem? These questions have not been empirically addressed, except by a recent study published in the new journal Information Visualization (IVS). In this strongly cognitive-flavored study, Ware et al. (2002) closely examined the role of good continuity in understanding a graph and began to explain why edge crossings are not only aesthetically unpopular, but they also slow down the speed of our visual search. Our perception seeks visual patterns no matter whether the perceived patterns are intended or not. When we see a group of nearby stars, we think of a constellation. When we see a few stars in a row, our mind draws a line to join them. Gestalt laws are concerned with our pattern perception. Gestalt laws determine whether we see something as a figure as opposed to ground. Ware’s book Information Visualization is a good source on this topic. The principle of good continuation is 198 Information Visualization one gestalt law that is especially relevant to graph drawing. It suggests that we will more easily see smooth continuous contours than jagged ones. It also suggests that we will be able to interpret graphs that use smoothly curved lines more easily than grid layout graphs, because the continuous lines are more likely to “pop out” as perceptually complete objects (Ware et al., 2002). As defined by Ware et al. (2002), a graph has a good continuity if multi-edge paths in the graph are drawn as straight as possible; in other words, if each path as a whole is close to a smooth line as much as possible. The dependent variable was the time to perceive the shortest path between two nodes specified in a spring layout graph. A regression model was built based on a number of independent variables of topological and aesthetic properties. The primary variables are summarized as follows: The total number of edge crossings in the entire graph was also recorded, mainly for comparison purpose as this measure was used by Purchase (2000). Test graphs contained 42 nodes and the degree of each node was randomized between 1 and 5. The length of a target shortest path was between 3 and 5. The findings are intriguing. The path length has the strongest effect on the perception time. Since the path length is determined by the graph itself, continuity is the graph drawing aesthetic property that has the strongest effect on finding shortest paths. Furthermore, the regression mode makes it possible to estimate cognitive costs associated with various factors. For example, 100 degrees of continuity on a path add 1.7 seconds to the time spent on finding the shortest path, and each edge crossing adds 0.65 seconds. This is equivalent to saying that the cognitive cost of a single edge crossing causes about the same amount of trouble as 38 degrees of continuity. As a result, a graph drawing algorithm may consider a new strategy for drawing easy to understand graphs by increasing the path continuity as well as reducing the number of edge crossings. The study also confirms that the total number of edge-crossings in the entire graph is not a significant indicator of response time. Instead, what really matters for this type of task is the number of edges that cross the shortest path itself. This is an intuitive finding. It should be noted that the shortest path tasks here are different from tasks tested by Purchase (2000). In the 2000 study, the tasks were identifying nodes and edges that hold a graph together as a single piece. It would be even more interesting if the two experiments were more comparable to each other. The Cleveland–McGill (1984) study and the Ware et al. (2002) study are different in many ways, but they have something fundamental in common – both of them established an interrelationship between perceptual tasks and the way that the information is displayed. Cleveland and McGill focused on the accuracy of perceptual • • • • • • Node continuity – given a node, the continuity is measured as the degrees of the angular deviation from a straight line of the two edges on the shortest path through the node. Path continuity – the sum of node continuity measures for all nodes on the path. The best possible path continuity is 0 degree. Number of crossings – the number of crossings on the shortest path. Average crossing angles – the average cosine angles of each edge crossing. Acute angles were expected to be more disruptive than more perpendicular angles. Number of branches – the degree of each node on the shortest path minus two, i.e. branches that are not part of the shortest path. Judging a path with more such branches can be expected to be more difficult. Length of the shortest path – self-explainatory. Empirical Studies of Information Visualization 199 tasks, whereas Ware et al. focused on the time efficiency. Cleveland and McGill studied tasks of comparing values, whereas Ware et al. studied tasks of finding shortest paths. 6.6 2D versus 3D It doesn’t seem to matter whether we perform our 3D tasks well or not; many of us like 3D visualizations anyway. Interestingly, many of us do not like 3D by instinct. Human beings are creatures who live in a 3D world. Many people enjoy 3D video games and immersive virtual reality, and yet many people equally enjoy the simplicity and clarity of 2D space. A 3D world gives us an extra degree of freedom, but sometimes the extra degree of freedom also leads to confusions and complications. Researchers are puzzled by the seeming mismatch between 3D’s appeal and 3D’s performance, as we shall see in the empirical studies explained in this section. Shneiderman (2003) recently questioned whether the richness of 3D reality is necessarily a good model after all. He quoted evidence such as disorientating navigation and annoying occlusions from the real world to highlight the potential pitfalls of 3D interface designs, and emphasized that to go beyond merely mimicking reality advanced designs we need to support the following three important features: • • • rapid situation awareness through effective overviews; reduced number of actions to accomplish tasks; and prompt, meaningful feedback for user actions. One of the practical questions is: how much of the 3D attraction can be translated into improved performance scores and more enjoyable experience? Designers may ask themselves whether they have turned over every stone, or more precisely, pushed the right buttons, used the right colors, the right shapes, and so on. The notion of Type I and Type II tasks in information visualization can be useful in organizing various empirical findings regarding the 2D versus 3D debates. Recall that Type I tasks are only concerned with visual–spatial properties, while Type II tasks involve a deeper understanding of the underlying meaning which may or may not be fully revealed by such visual–spatial properties. An increasing number of empirical studies began to address the potential gains by the extra dimension of display. We will see quite a few of them in this section. Do we have sufficient empirical evidence to settle the 2D versus 3D debates? Is a 3D visualization always better than its 2D counterpart? Is this merely a question about an extra dimension of freedom? The following empirical studies illustrate some of the major concerns, what has been done so far, and what might be done in the future. 6.6.1 Spatial versus Symbolic References In an earlier study Jones and Dumais (1986) assessed the accuracy of spatial versus symbolic references in three experiments of news article reading and filing. In the first experiment, performances with spatial references were not only poor in comparison to the symbolic references, but also deteriorated more rapidly than symbolic references as the number of articles increased. In the second experiment, a 2D space was used with an office metaphor, induced by landmarks such as a desk, a table, and 200 Information Visualization filing cabinets. In the third experiment, subjects placed objects in an actual 3D mock office. However, neither of these enhancements significantly improved the accuracy of spatial reference; in fact, the performance remained below what was achieved by symbolic references in the first experiment. Jones and Dumais suggested that the formation of spatial memory in many situations is essentially effortless; however, this is not necessarily the case in a computer filing situation. Furthermore, because spatial references with a computer seemed to entail higher costs than symbolic references, and the immediate benefits of such high-cost actions may not always be clear, users may choose symbolic references over the spatial ones. They also suggested that news articles filed in the experiments do not especially lend themselves to a spatial organization. Given the popularity of spatial models and their applications to visualizing news article collections and webpages a decade after these experiments, for example, ThemeView and Data Mountain, this is indeed a challenging empirical question that the information visualization community must answer. 6.6.2 Understanding 3D Structures Westerman and Cribbin (2000) examined information search tasks in 2D and 3D multidimensional scaling (MDS) maps and particularly focused on the extent to which 2D and 3D MDS maps differ in terms of semantic richness and cognitive demands. They compared the amount of additional semantic information that a 3D MDS map conveys with the increased cognitive demands, and concluded that the gain is insignificant as compared to the loss. From Cleveland and McGill’s study of the elementary perceptual tasks, we know that some tasks tend to be performed more accurately than others, depending on the way quantitative information is depicted. When dealing with MDS maps, a basic perceptual task is to judge the distance between two objects. Judging such distances in MDS can be a challenging task, especially in 3D MDS maps. This leads to the question about graphs: is this low cost-effect ratio also true for 2D and 3D graphs, where the distance between two objects is represented by explicit links? The first quantitative estimate of the benefits of 3D stereo viewing for perceiving graphs was made by Ware and Franck (1996) in their study of how people understand 3D graphs presented in 2D and 3D displays. They found that people performed significantly better in 3D than in 2D conditions. The task was to determine if there was a path of length 2 between two highlighted nodes in a graph (Figure 6.10). In Ware and Frank’s study, the 3D condition allowed users to move and rotate the graph. Indeed, motion and rotation may be the key to the superior 3D performance. 6.6.3 Data Mountain Robertson et al. (1998) described the design of Data Mountain, an inclined plane in a 3D desktop virtual environment for users to place icons of documents at arbitrary positions in the plane. Data Mountain uses a manual layout to exploit spatial memory for long-term use. The Data Mountain study also reminds us of the Jones–Dumais study of spatial versus symbolic references, such as the spatial placement of document icons and the reference to the spatial memory of users. Empirical Studies of Information Visualization 201 Figure 6.10 Graph containing 78 nodes and 104 edges. The subject’s task was to determine if there was a path of length 2 between the two red nodes (Ware and Franck, 1996). © 1996 ACM, Inc. Reprinted with permission. Data Mountain uses a plane tilted at 65 degrees for a user to place web pages anywhere on the mountain. The landscape texture in Data Mountain provides passive landmark references, but the landmarks themselves have no special meaning. The design of Data Mountain values the serendipity of its layouts: as long as a layout means something to whoever created it, anything goes. The empirical evidence suggested that it took less time to retrieve web pages from Data Mountain than from a standard tree browser. The improvement was attributed to the provision of both spatial and symbolic cues in Data Mountain. With reference to the Jones– Dumais study (1986), this is probably the strongest positive evidence for a combination of spatial and symbolic references. Cockburn and McKenzie conducted a series of empirical studies concerning the effects of 2D and 3D interfaces, especially with designs that closely mimic Data Mountain. Two of their recent studies are of particular interest. Cockburn and McKenzie (2001) compared the effects of 2D and 3D Data Mountain-like interfaces for placing and retrieving webpage thumbnails. Two independent variables were included: (1) interface type – 2D and 3D, and (2) data density – sparse, medium, and dense, implemented as 33-, 66- and 99-thumbnail displays respectively. Subjects placed webpage icons through Data Mountain interfaces and retrieved these webpages. The mean task completion time was measured as the dependent variable. There were no statistically significant differences between the 2D and 3D interfaces, but there was a significant preference for the 3D interfaces. On the other hand, the main effect of data density was statistically significant, and there was no interaction between the two factors. In an even more interesting study, Cockburn and McKenzie (2002) compared not only the dimensionality factor, but also a realism factor, which was a rare effort in empirical studies of visualization interfaces. They used a 2 3 3 mixed factorial analysis of variance (ANOVA). The dimensionality factor has three levels: 2D, 21⁄2D, 202 Information Visualization Figure 6.11 Physical and virtual interfaces used by Cockburn and McKenzie (2002). © 2002 ACM, Inc. Reprinted with permission. and 3D, whereas the realism factor is between physical and virtual versions. Fishing lines were used in their physical interfaces to hold webpage printouts in place. Data density was once again included, like their 2001 experiment: sparse, medium, and dense. A total of six interfaces were used in their experiment (Figure 6.11). Although the empirical studies discussed so far in this section used the term spatial memory, they were not related to psychometric tests of spatial memory. In the second half of the chapter, we will see empirical studies that specifically measured spatial memory as a cognitive ability. The second Cockburn–McKenzie study found the main effect of dimensionality – it took increasingly longer to retrieve pages from interfaces of higher dimensions. The results were confirmed by the decreasing effectiveness reported by the subjects. In summary, empirical evidence appears to suggest that simply increasing an interface from 2D to 3D is unlikely to be enough to boost the task performance unless additional functions are provided so that users can have greater controls of objects in 3D interfaces. 6.7 Cognitive Abilities Take a look at a user satisfaction survey of an empirical study of information visualization, and you will quickly find out that there will always be some users who will reportedly like the spatial metaphor of an interface, and there will always be others who will tell you they don’t. A more interesting fact is that sometimes our preferences appear to have little to do with how well or how badly we tend to perform with our favorite interfaces anyhow. In contrast, there are indeed situations in which we excel if we have in hand our favorite tools. How do we know what we will be good at? And if we don’t know, how do we find some ways to compensate our potentially disadvantaged cognitive appetite? These are the questions that research in individual differences can help us to answer. Individual differences have been a unique area of study in such disciplines as psychology and human computer interaction. Individual differences refer to Empirical Studies of Information Visualization 203 relatively stable personal and cognitive traits. It is believed that a wide range of cognitive abilities of individuals can be accounted for by a relatively fewer number of cognitive factors. Some factors may be more relevant to information visualization than others. For example, spatial ability, spatial memory, and associative memory are among the ones that are more likely to be relevant. Individual differences are just one aspect of the dynamics of using information visualization and virtual environments. Social and ecological dimensions are equally important in understanding how people perceive and behave in a virtual environment. These dynamics in virtual environments will be discussed in Chapter 7. There are two general views on individual differences. One believes that the differences can be reduced through education and training, while the other believes that these differences are difficult to change, but may be accommodated through the use of specially designed tools. The 2000 special issue on Individual Differences in Virtual Environments in the Journal of the American Society for Information Science (Chen et al., 2000) features a number of articles on this topic. Empirical evidence in situations similar to the use of information visualization systems suggests that spatial ability, associative memory, and visual memory should be a good starting point (Benyon, 1993; Carroll, 1993; Dahlbäck et al., 1996; Dillon and Watson, 1996; Höök et al., 1996; Vicente and Williges, 1988). The first example is an empirical study focusing on spatial ability and visual information retrieval. The second example is focusing on associative memory and visual memory, and the third is the most extensive experimental study of the three, focusing on the role of both spatial ability and associative memory in visual information retrieval. 6.7.1 Cognitive Factors Methodologically, an introduction to factor-referenced cognitive tests in Eckstrom et al. (1976) is the first step. Is it sensible to test an information visualization design with the whole range of cognitive factors? Is it a practical starting point to establish a baseline of representative information visualization design? Can factor-referenced cognitive tests provide researchers and designers with common ground for analyzing and comparing the effects of different information visualization options in comparable cognitive tests? The first comprehensive introduction of individual differences into human– computer interaction (HCI) is Egan’s seminal work (1988), which has inspired many studies in the field. According to Egan (1988), differences between users can be in the staggering order of 20:1 for common computing tasks, such as programming and text editing. More importantly, such differences can be understood and predicted, as well as modified through design. The most influential and updated work on the study of individual differences in psychology is presented by Carroll (1993). Cognitive factors are grouped into a threelevel hierarchy: general intelligence (g) is at the top level,eight ability categories reside at the second level, and first order factors derived from these general ability types are at the third level. In particular, the eight second-level general ability categories are: • • • • crystallized intelligence fluid intelligence general memory and learning broad visual perception 204 Information Visualization • • • • broad auditory perception broad retrieval ability broad cognitive speed processing speed. At the third level, each of these general ability types is split into first order factors; for example, memory becomes associative memory, visual memory, episodic memory, and memory span. This hierarchical organization is regarded as the standard conceptualization (Kline, 1994; Dillon and Watson, 1996). Dillon and Watson (1996) presented a thought-provoking review of the study of individual differences, and its position in the field of HCI. According to Dillon and Watson, a core number of basic cognitive abilities have been reliably and validly identified, as influencing the performance of specific tasks in predictable ways. Several areas are potentially fruitful for HCI. They recommend that psychological measures of individual differences should be used as a basis for establishing context, and achieving a greater degree of generalisability of HCI findings. The subsequent empirical studies investigate the relationship between individual differences and information foraging within a semantically organized virtual world. This virtual world is constructed automatically, based on the results of information visualization. Three cognitive factors are specifically examined in the use of the semantically organized virtual world: spatial ability, associative memory, and visual memory. Studies of these not only illustrate a generic methodology for evaluating the usefulness of information visualization design, but also highlight cognitive factors that may lead to insights into information visualization design itself. Spatial Ability Spatial ability, also known as visualization ability, is the ability of an individual to manipulate or transform the image of spatial patterns into other arrangements (Eckstrom et al., 1976). The role of spatial ability in navigating through information structures has received much attention over the past decade, ranging from large file structures and database systems, to hypermedia, and virtual reality-based spatial models. For example, Vicente and Williges (1988) found that spatial ability affected the user’s ability to navigate a large file structure. Campagnoni and Ehrlich (1989) reported that users with good visualization ability used the top-level table of contents less frequently than users with lower visualization ability, suggesting that a good spatial ability may help in memorizing how the information is organized. Dahlbäck et al. (1996) found a strong correlation between users’ spatial abilities and their task completion times. The fastest subject completed tasks 19 times faster than the slowest subject. The study suggested that solving problems in the real world and solving problems in a virtual world involve different spatial abilities. Our 1996 meta-analysis of empirical studies of hypertext found a number of large effects that are relevant to cognitive abilities (Chen and Rada, 1996). The metaanalysis found that the complexity of tasks has the greatest effect size on effectiveness measures, such as accuracy and error rate measures (r 0.63), followed by graphical maps (r 0.38). More interestingly, the strongest effects on efficiency measures are due to spatial ability of individuals (r 0.45) and the complexity of tasks (r 0.58). These are large effect sizes according to the guidelines of Cohen (1977) – one should take these factors seriously. Empirical Studies of Information Visualization 205 The relationship between spatial ability and visual navigation in a virtual realitybased spatial user interface was studied by Chen and Czerwinski (1997). Spatial ability, measured by paper-folding tests (VZ-2) included in Eckstrom et al. (1976), was strongly correlated with the accuracy of sketches made by subjects after they searched through a semantically organized spatial model. The spatial ability was positively correlated with the differences between the main structure in the spatial layout and the sketches made by individuals (Spearman’s r 0.774, p 0.004, onetailed). Similarly, a strong correlation was found between spatial ability and the secondary structures in the spatial layout, and structures memorized by individuals (Spearman’s r 0.591, p 0.036, one-tailed). Associative Memory Sometimes we learn two things together, despite they have nothing to do with each other otherwise. We may associate one thing with another in this way. Associative memory is our ability to recall one part of the relationship when seeing the other part (Carroll, 1974). This factor involves the storage and retrieval of information from intermediate-term memory. Individual differences observed in such conditions may be largely due to the successful use of strategies such as rehearsal, and using mnemonic mediators. In the following examples, a document in a spatial model of a semantic network is shown as a colored sphere labeled by the initials of authors. An interesting question is what role is played by associative memory in visual navigation. Can we build a mental map based on spatially associated graphical and textual cues? If we can develop an effective mental map, presumably we should be able to find the information more efficiently. Visual Memory Visual memory is the ability to remember the configuration, location, and orientation of figural material, another potentially useful cognitive factor for the use and evaluation of a virtual environment, based on a spatial metaphor (Eckstrom et al., 1976). Visual memory involves different cognitive processes from those used in other memory factors. A good visual memory should enable users to memorize and locate local structures more efficiently; thus more effective information search and information foraging is possible. The three empirical studies summarized below tested the effects of individual differences on visual search and navigation. Individual differences were measured by factor-referenced cognitive tests (Eckstrom et al., 1976). Spatial ability is measured by VZ-2 scores, associative memory by MA-1 scores, and visual memory by MV-1 scores. The search space consisted of a document similarity network of 169 papers published in three ACM SIGCHI conference proceedings between 1995 and 1997. The document similarity networks were in fact Pathfinder networks derived from interdocument similarities determined by latent semantic indexing (LSI) (Deerwester et al., 1990). The document network was rendered in virtual reality modeling language (VRML) 2.0. Users can walk and fly through the semantic space. The screen is split into two frames: the semantic space is displayed as a virtual world in the left-hand side frame. Document spheres were color coded by the year of their publication: 1995 206 Information Visualization papers in red, 1996 in green, and 1997 in blue. Each sphere is labeled with the initials of authors of the paper. Clicking on the sphere will load the abstract into the righthand side frame. The technical background for LSI and Pathfinder network scaling is described in Chapter 2. Readers may also consult the original publications such as Chen (1998a,b) and Chen and Czerwinski (1998). 6.7.2 Study I: Spatial Ability The first study focuses on correlation relationships between spatial ability and visual navigation. Eleven subjects performed search tasks through the VRMLbased interface. Moving the mouse cursor over a sphere would pop up the paper’s title. Clicking on the sphere would display the abstract in the right-hand side frame. All the movements on the computer screen were videotaped and subsequently analyzed. Users can zoom in and out to obtain an overview of the entire structure or a detailed view of specific areas. The user interface also allows teleporting-like instant zoom right in to the front of a sphere. Subjects’ spatial ability scores were collected through a standard paper-folding test (Eckstrom et al., 1976). The paper-folding test requires subjects to answer multiple-choice questions about the consequence of punching a hole in a paper folded in a particular way. A spatial ability score ranges from 0 to 20. This study has the average score of 10 and a standard deviation of 3.91. The tasks were finding papers on particular topics and saving relevant papers to a local directory on their PC. In the first task, subjects were told to find as many papers as they could about a topic, whereas in the second task, they were told to find the five most relevant papers on a different topic. The 169 papers fall into three categories of relevancy: (1) the most obvious ones – their relevance is clear from their titles; (2) the intermediate ones – their relevance is not clear from their titles alone, but clear enough if you read their abstracts; and (3) the least obvious ones – reading the full text is necessary. For example, if we look for papers on visualization, it is immediately clear by just reading the titles that papers 1 and 2 below are relevant, but the relevance of paper 3 is less obvious: There were 24 relevant papers for Task 1, and 18 for Task 2. Having completed Task 1, subjects sketched the structure of the visualized semantic space as best they could from memory. Research has shown that if we engage ourselves more deeply in the study of an object, then we tend to recall more details (Craik and Lockhart, 1972). In this case, since subjects engaged in visual search, such sketches should reveal to which parts of the semantic space they directed most attention. Subjects performed a categorization and abstraction task after Task 2 – they gave names to clusters of papers in the visualized semantic space. This task was designed to find out whether subjects could summarize groups of papers associated with distinctive structural patterns, and what naming schemes they would use. Such input can be used as semantic labels, landmarks, and signs. However, some subjects found this task very difficult; some could not even complete it. Some wanted to check particular spheres again before they could provide a name for the given structures. • • • Paper 1. Tilebars: visualization of term distribution information in full-text information access. Paper 2.Visualizing complex hypermedia networks through multiple hierarchical views. Paper 3. An organic user interface for searching citation links. Empirical Studies of Information Visualization 207 Task performance scores were measured as the number of times that a subject read an abstract, the number of saved abstracts, and the number of relevant abstracts saved. The entire session lasted approximately 30 minutes. Task performance scores were split into a high spatial ability group (A) and a low spatial ability group (B). The high spatial ability group saved more than twice as many abstracts as the low spatial ability group in Task 1, and slightly more in Task 2. Group A also found about twice as many relevant abstracts as Group B. Spatial ability was positively correlated with recall scores in both Task 1 and Task 2 (r 0.42 and 0.37, respectively), but were negatively correlated with precision scores in Task 1, and to a lesser degree in Task 2 (r 0.53 and 0.18, respectively). The total number of abstracts viewed was not correlated with spatial ability in Task 1 (r 0.07), but negatively correlated in Task 2 (r 0.43). The number of saved abstracts was positively correlated with spatial ability in both Task 1 and Task 2 (r 0.45 and 0.27, respectively). The results suggested that subjects did well if relevant papers were located in structurally significant areas of the user interface, especially at nodes with high degrees. However, subjects were less successful on the outskirts of the structure. Subjects typically examined key positions such as branching points or central points in their first few moves. The videotapes recorded a number of interesting search strategies. First, most subjects started their search from the central circle structure, and they ignored the outskirts of the central circle during their initial search. Next, subjects would check a number of positions on the circle, especially points connecting to branches. Over time, they would gradually expand their search space outwards, to reach nodes farther away from the central area.An interesting observation was that after a few visits to a target document, subjects appeared to suddenly decide the document was relevant after all and would save it immediately. This implies that subjects continuously adjust their relevance criteria. Some subjects hopped from one cluster to another in long jumps, whereas others carefully examined each node along a path, according to the virtual semantic structure. Subjects who made longer jumps apparently realized that they might be able to rely on the structural patterns to help with their navigation. Navigational patterns also highlighted the special role of distinctive structural patterns such as circles, stars, and long spikes, as we expected from earlier research (Chen, 1998a, b). Some people only realized that they could benefit from the structure during the second half of their session. The spatial memory test highlighted the need for reinforcing strategically significant points, or structural hotspots, as well as larger structural patterns in the virtual environment. Strong stimuli (e.g. landmarks or signs) should be recommended, to reinforce users’ cognitive map of the virtual space. For example, an animation of how papers were organized would help users to understand the nature of the organization. This notion awaits further user interface design work. Correlation between spatial ability and user behavior was computed for a number of different tasks. Although we found that recall was positively correlated with spatial ability, as were a number of other measures, the overall impact of spatial ability was not straightforward. Sometimes the direction of the correlation was unexpected. A few aspects of the design of this study could be improved in future research, to help clarify the impact that spatial ability might have on the usability of such information visualizations. For example, the entire task session was very limited in terms of time, especially for subjects who had not used the VRML viewer prior to the test session. The sample size should be increased to minimize variability in the data resulting from extreme combinations of spatial ability and experience with computers. 208 Information Visualization On the other hand, the spatial memory test and the categorization task turned out to be very informative.We recommend usability studies on visualization-based information systems to include such tasks. 6.7.3 Study II: Associative Memory and Visual Memory The second study focuses on the effects of associative memory and visual memory on visual navigation performance with the same interface as in Study I. Ten subjects participated in this study. Associative memory scores (MA-1) and visual memory scores (MV-1) were obtained one day before search sessions. Subjects sketched the spatial layout of the search space at the end of the first block of search tasks. When subjects completed the second block of tasks, they were asked to name the cluster of papers in the semantic space. The recall and precision scores were calculated based on the underlying latent semantic indexing techniques. We used the keywords appearing in the task descriptions to formulate the search. The top 20 papers returned by the LSI were regarded as the short-listed documents for the given topics. Because the semantic space was generated on the basis of the LSI modeling, it is reasonable to use the results of LSI on the same document collection to measure the relevance. Correlations were computed between task performances and subjects’ memory scores. The number of abstracts saved by each individual was positively correlated with the memory associated test in Task 1. As predicted, subjects with better memory performed better. However, the number of saved abstracts was negatively correlated with both memory tests in Task 2. The possible reason for this could be that in Task 2 it was difficult to search for a specific topic. Subjects would need to explore the content of the paper more deeply, especially those without background in this area. The use of more general content papers could solve this problem. Associative memory was strongly correlated with the mean recall scores of Task 1 (r 0.855, p 0.003), whereas visual memory was negatively correlated with the mean recall scores of Task 2 (r 0.649, p 0.041). All the subjects included a central circle in their sketches. However, the detailed structures vary. Figure 6.12 includes four sketches of the spatial layout of the underlying search space, made by subjects who had the highest performance scores, as well as the ones who had lowest performance scores. In sketch (a), the subject who scored high in both recall and precision accurately depicted most details about the surrounding branches and strokes inside the central circle. The sketch in (b) was very interesting: most links were omitted, but it still gave an accurate outline of the structure. This sketch shows the branches that the subject visited several times in greater detail. In sketch (c), although the overall recall was not as accurate as (a) and (b), the subject depicted the branches that he searched, especially the branch he started with. Finally, sketch (d) was the least accurate by a subject with the lowest recall score for the first topic. With little knowledge of the underlying structure, many users adopted a bruteforce search strategy; much of their initial inspection relied on mouse hovering and title pop-ups. People with stronger memory abilities were expected to perform better. In fact, a strong positive correlation was found between associative memory and task performance for Task 1 (r 0.855, p 0.003). However, visual memory was negatively correlated for Task 2 (r 0.649, p 0.041). Empirical Studies of Information Visualization 209 Figure 6.12 Subjects’ sketches of the semantic space they searched. Source: Chen and Czerwinski (1997). 6.7.4 Study III: Associative Memory and Spatial Ability The third study further explores the effects of cognitive factors on visual navigation performance. It was a within-subject design. Scores of spatial ability (VZ-2) and associative memory (MA-1) were obtained from a pretest. Two user interfaces were used in each session: one was spatial, and the other was textual. The search task was to find as many papers as possible relevant to four topics. Twelve subjects were scheduled according to a Latin-Square design. Subjects searched in one user interface for two topics, and then switched over to the other interface for the remaining two topics. Ten minutes were allocated for each topic. To compensate for the less familiar search topics, the relevance of a document was calculated based on pooled answers among subjects. If two or more subjects marked a document as relevant to a given topic, then it would be treated as relevant. The recall and precision were calculated as usual. Spearman’s correlation coefficients were computed between subjective ratings, and both spatial ability and associative memory scores. Unless stated otherwise, all the statistical significance was one-tailed at the conventional 0.05 level. Higher spatial ability scores and higher associative memory scores were associated with the positive direction. Spearman’s correlation coefficients reveal a negative correlation between associative memory and the usefulness rating on the textual user interface (Spearman’s r 0.597, p 0.020), implying that a user with a good associative memory is more likely to favor the spatial user interface, rather than the textual one. There was a 210 Information Visualization positive correlation between spatial ability and associative memory (Pearson’s r 0.581, p 0.024). A negative correlation was found between associative memory and precision in the spatial user interface (Pearson’s r 0.544, p 0.034), and a negative correlation between spatial ability and precision in the textual user interface (Pearson’s r 0.553, p 0.031). Both correlations are statistically significant, and the magnitudes can be regarded as medium or large. In general, subjects performed slightly better with the textual interface than the spatial one. General factorial analysis and linear regression analysis were used to analyze the multivariate effects on the overall information foraging performance. Spatial ability scores (VZ-2) and associative memory scores (MA-1) were used as covariants in these models. A dummy variable S was also assigned, to indicate whether the spatial user interface was used before the textual one. There was a strong effect of associative memory on precision (F(1, 10) 6.69, p 0.027), which explains 40% of the variance. However, the main effect of associative memory on recall was not statistically significant (F(1, 10) 0.672, p 0.431). Stepwise linear regression analysis yielded a single predictor model for both precision and recall. Associative memory explained more than 70% of the variance (R2 0.731, Adj. R2 0.707). Spatial ability (VZ-2) and the dummy variable S were excluded from the model. Similarly, associative memory remained the best predictor for recall, explaining 59% of the variance according to the adjusted R2. In summary, associative memory appears to be a good predictor of task performance with the spatial user interface, whereas spatial ability seems to be overshadowed by the cognitive factor of memory. The first study found a strong correlation between spatial ability and the accuracy of a sketch task. The second study extended its scope to the two cognitive factors, associative memory (MA-1) and visual memory (MV-1). A strong positive correlation was found between associative memory and overall recall. Finally, the third study compared the same spatial user interface and a textual user interface in a within-subject Latin-Square experimental design. Precision with the spatial user interface was negatively correlated with associative memory, whereas precision with the textual user interface was negatively correlated with spatial ability. 6.8 Summary In this chapter, we have discussed a variety of empirical studies concerning various aspects of information visualization, ranging from the usage of classic information visualization designs such as cone trees and hyperbolic view browsers to individual differences in terms of spatial ability, associative memory, and visual memory. Much of the existing empirical studies can be divided into ones that deal with lower-level elementary perceptual tasks or higher-level application-related tasks. Although information visualization can no doubt benefit from all empirical studies, the crucial and profound understanding at this stage is more likely to come from the study of lower-level tasks. Many application-oriented empirical studies have generated valuable insights into the complex relationship between tasks and visual cues. On the other hand, the empirical link between visual attributes and perceptual tasks is still missing in many areas of information visualization.
登録:
投稿 (Atom)
