User experiments with five tree visual

A. Kobsa, "User Experiments with Tree Visualization Systems," in IEEE Symposium on Information Visualization, Austin, TX USA, 2004. (Link)

An empirical study that compared the 5 infovis systems for tree hierachies with using Windows Explorer as baseline system. The five infovis systems are

1. Treemap 3.2
2. Sequoia View 1.3
3. BeanTrees
4. Star Tree Studio 3.0
5. Tree Viewer

The tasks in this study included 15 questions that could be divided into two catgories: structure-related tasks and attribute-related tasks.

This study find the baseline--Windows Explorer, works well in terms of task correctness, completion times, and user satisfaction. In the five tested systems, Treemap 3.2 has achieved similar or better performace than Explorer. The other four were overperformanced by Explorer. Bean Trees had relatively the worse performance based on the three evaluation metrics.

The author also used video records of screen shot for further analysis, and gave detailed descriptions of each tested visual system.

Strengthes:

        1. A true emprical studies of usability and usefulness of infovis systems.

        2. Strong study design, based on both previous study in InfoVis 2003 Contest and their own requiements.

        3. Comprehensive analyses based on quantitative metrics and qualitative analysis.

        4. Interesting results: system 2-5 had poor performance compared with Windows Explorer, which I didn't expected before reaching the results section. 

Weaknesses:

        As theoretical studies like to view InfoVis in two perspectives: Visual representation and interaction, I at first thought this paper would compare the unique feature of each visual representation of hierarchies in tree structure. The results of this study, however, mixed the two features. It is hard to examine whether the performance of each system comes from the unique visual feature, or the having or lacking of proper interactive features to support the visual representation. I wonder what the different impacts between visual representation of each system and the interactive functions of each system are; or, for each visual representation, what interactive functions can help to improve performance most than other functions. At the end, the author suggested in the conclusion section that further studies might look at the functionality beyond the pure visualization, which could help increase the performance of systems.