%%%%%%%%%%%%%%%%%%%%%%% file template.tex %%%%%%%%%%%%%%%%%%%%%%%%%
%
% This is a general template file for the LaTeX package SVJour3
% for Springer journals.          Springer Heidelberg 2006/03/15
%
% Copy it to a new file with a new name and use it as the basis
% for your article. Delete % signs as needed.
%
% This template includes a few options for different layouts and
% content for various journals. Please consult a previous issue of
% your journal as needed.
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% First comes an example EPS file -- just ignore it and
% proceed on the \documentclass line
% your LaTeX will extract the file if required
\begin{filecontents*}{example.eps}
%!PS-Adobe-3.0 EPSF-3.0
%%BoundingBox: 19 19 221 221
%%CreationDate: Mon Sep 29 1997
%%Creator: programmed by hand (JK)
%%EndComments
gsave
newpath
  20 20 moveto
  20 220 lineto
  220 220 lineto
  220 20 lineto
closepath
2 setlinewidth
gsave
  .4 setgray fill
grestore
stroke
grestore
\end{filecontents*}
%
\documentclass{svjour3}                     % onecolumn (standard format)
%\documentclass[smallextended]{svjour3}     % onecolumn (second format)
%\documentclass[twocolumn]{svjour3}         % twocolumn
%
\smartqed  % flush right qed marks, e.g. at end of proof
%
\usepackage{graphicx}
\usepackage{natbib}
%
% \usepackage{mathptmx}      % use Times fonts if available on your TeX system
%
% insert here the call for the packages your document requires
%\usepackage{latexsym}
% etc.
%
% please place your own definitions here and don't use \def but
% \newcommand{}{}
%
% Insert the name of "your journal" with
 \journalname{Virtual Reality}
%
\begin{document}

\title{Towards the Design of Transitional Interfaces: %\thanks{Grants or other notes
%about the article that should go on the front page should be
%placed here. General acknowledgments should be placed at the end of the article.}
}
\subtitle{An Exploratory Study in a Semi-Immersive Hybrid User Interface}

%\titlerunning{Short form of title}        % if too long for running head

\author{Felipe G. Carvalho        \and
        Daniela G. Trevisan \and
        Alberto Raposo %etc.
}

%\authorrunning{Short form of author list} % if too long for running head

\institute{Felipe G. Carvalho \at
              Pontifical Catholic University of Rio de Janeiro - PUC-Rio \\
              Computer Graphics Technology Group - Tecgraf - Rio de Janeiro - Brazil \\
              Tel.: +55 21 2512-5984\\
              Fax: +55 21 3527-1848 \\
              \email{kamel@tecgraf.puc-rio.br }           %  \\
%             \emph{Present address:} of F. Author  %  if needed
           \and
           Daniela G. Trevisan \at
              Federal Fluminense University \\
              Computer Institute - Niteroi, Rio de Janeiro, Brazil \\
              Tel.: +55 21 2629-5675\\
              \email{daniela@ic.uff.br} 
           \and
           Alberto Raposo \at
              Pontifical Catholic University of Rio de Janeiro - PUC-Rio \\
              Department of Informatics\\
              Tel.: +55 21 3527-1500\\
              Fax: +55 21 3527-1848 \\
              \email{abraposo@inf.puc-rio.br} 
           }

\date{Received: date / Accepted: date}
% The correct dates will be entered by the editor


\maketitle

\begin{abstract}
A task which can be decomposed into subtasks with different technological demands may be a challenge, since it requires multiple interactive environments, as well as transitions between them. Some of these transitions may require changes of hardware devices and paradigms of interfaces at the same time. Some previous works have proposed various setups for hybrid user interfaces, but none of them was mainly focused on the design of   transition interactions.  Following this direction our work brings together the importance of interaction continuity as a guideline to help the design and evaluation of transitional interfaces within a Hybrid User Interface (HUI). Finally, an exploratory study demonstrates how this design aspect is perceived by the users during transitions in an HUI composed by three interactive environments.
\keywords{transitional interfaces \and hybrid user interfaces \and continuity properties}
% \PACS{PACS code1 \and PACS code2 \and more}
% \subclass{MSC code1 \and MSC code2 \and more}
\end{abstract}

\section{Introduction}
The evolution of human-computer interaction has led to the development of different interfaces and interaction mechanisms which are progressively being combined to create richer forms of interactions. The support to tasks that require the use of different interfaces has become increasingly evident with the emergence of fields of study such as Hybrid User Interfaces (HUI)~\citep{Feiner91}. In this field, efforts are being joined in order to harmoniously blend both virtual (applications, interaction techniques, graphics, etc.) and physical (the input and output) elements from one or more interfaces.  The design of HUIs may thus result in new interfaces with hybrid resources aiming to integrate previously separate functions.

The way in which hardware and software elements are integrated into an HUI may demand interface mechanisms or a kind of interface that connects others, e.g. a form of transition interface or \textit{transitional interface} ~\citep{Billinghurst01,GrassetLB06}. 

The concept of transitional interfaces was first introduced by the MagicBook project ~\citep{Billinghurst01} through an application that moved the user seamlessly along the mixed reality continuum ~\citep{MC99}. In the MagicBook, the user walks from the real world to virtual reality passing through augmented reality using a hand-held device. The transition between these environments consisted of automated virtual camera movements and image effects, such as fade in and out.

A few years later, the concept of transitional interface was formalized aiming at a generalization of this term to multiple contexts ~\citep{GrassetLB06}. The meaning of context somehow encompassed the idea of 3D spaces with properties such as scale (macro, micro, nano), representation (photorealistic, cartoon, etc.) and others. For each context, there is a clear definition of a motion function related to the properties of a virtual camera, and, for each transition between these contexts, there is a transition function related to visual factors in order to continuously provide smooth transitional motion rather than teleportation. The hardware and software setups used by authors during the development of these ideas were based on the contexts of Virtual Reality and Augmented Reality, and a constant set of input devices and displays during the studies, such as a hand-held device ~\citep{MC99} or a Head Mounted Display combined with a 3D tracking input device ~\citep{GrassetDB08}. We noted that there was no change of interface paradigm during transitions since the contexts were restricted to 3D environments and interactions of transitions were conceived in an intuitive way. 

In this work, we claim that the range of action of a transitional interface may actually be larger than the mixed reality continuum. \citet{Molina08} proposed the mixed virtuality continuum as an extension of the continuum proposed by \citet{MC99}. In this continuum, before Virtual Reality (VR) interfaces, there is a series of intermediary interfaces ranging from 1D to pure 3D interfaces, passing through 2D graphical interfaces, 2D desktop, 3D desktop, 3D rendering of 2D interfaces, among others. This mixed virtuality somehow places those interfaces in the context of the evolution of the human-computer interaction. Altogether, they consist of a diversity of hardware and software settings that can certainly be combined to create hybrid environments, e.g HUIs (Fig.~\ref{fig:continuum}). Actually, it makes sense to think of HUIs and transitional interfaces as complementary subjects.

\begin{figure}[hbt]
\centering
%\includegraphics[bb=0 -25 85mm 55mm]{HybridDesk_Esquema_Envs.png}
%\includegraphics[width=0.90\textwidth]{Continuums_HEnvs.png}
\includegraphics[width=0.90\textwidth]{Continuums_HEnvs.eps}
\caption{Relationship between Mixed Continuums, Hybrid Environment and Transitional Interfaces.}
\label{fig:continuum}
\end{figure}

Considering this vision of HUIs, the concept of transitional interface can also be extended. The transitions would be placed between contexts that include an interface (of the mixed virtuality or mixed reality continuum) and the input devices and displays used (Fig.~\ref{fig:continuum}). The idea of motion function in this expanded vision would be seen as a function describing which input devices, interaction techniques, and displays are used at a given time. On the other hand, a transition function includes aspects such as procedures to change or handle devices as well as the use of graphic and audio effects to prepare for the motion function of the next context. It is important to note that this paper will use the term ``interactive environment'' with the same meaning as ``context'' defined hereby.

The main goal of this work consists in exploring the continuity properties (see Section~\ref{sec:continuity}) of transition interactions between interactive environments. We discuss the use of such design aspects as a guideline to help the process of evaluation and to reach smoother transitions within a hybrid interactive system. In order to achieve such goal, a semi-immersive system (called HybridDesk) composed of three interactive environments was developed, and transitions between these environments were created to help the accomplishment of a 3D annotation task in an oil and gas scenario application.


The remainder of the paper is structured as follows. Section 2 reviews relevant work in the areas of Hybrid User Interface and Transitional Interfaces. Interaction continuity as a design aspect is discussed in Section 3. The HybridDesk prototype is introduced in Section 4 as a practical example to the analysis of the design issues presented. The issues related to the conception of transitional interfaces within HybridDesk are presented in Section 5.  Section 6 reviews an exploratory evaluation carried out with users in order to provide content for the interaction analysis. Finally, Section 7 presents conclusions and discusses future directions of this work. 

\section{Related Work}

Interface systems that attempt to combine hardware and software components of different natures are sometimes called Hybrid User Interfaces, Mixed Reality Systems, or, until recently, Hybrid Display Systems. They are characterized by the use of multiple elements, which can be multiple devices or multiple user interfaces. The term HUI was first proposed by ~\citet{Feiner91}, referring to a heterogeneous environment, rich in interaction techniques, and with different kinds of devices used in a complementary way. 
   
A pioneer work in this area was the Office of the Future~\citep{Rascar98}, which combined several computer vision and computer graphics techniques to analyze surfaces of the real world and then add virtual information that was projected on them. 

\citet{RekiSaitoh99} explored the HUI heterogeneity using several computers and displays (projections and notebooks) in the same work environment. The EMMIE project~\citep{Butz99} developed a system designed for collaboration that was similar to the work of Rekimoto, but included the use of Augmented Reality (AR) with a see-through HMD (Head Mounted Display).

\citet{Nakashima05} presented a prototype of a collaborative work environment with 2D and 3D environments for graphic modeling tasks. The environment uses a display called IllusionHole for 3D interactions, while 2D interactions are supported by a projection. The lack of transitions between those environments could be explained by the fact that both of them work in the same 3D task using different interactions techniques. This same explanation could be used for an HUI for manipulation of medical data, also using 2D and 3D interactions, which was developed by ~\citet{Bornik06}: a single 3D pointer is used as an interaction tool, and two visualization forms are available, one on a tablet and another on a projection.  

\citet{Benko05} created a hybrid environment composed of an LCD (Liquid Cristal Display) placed vertically, a touchable display placed horizontally, and a see-through HMD for AR. This environment is used for the manipulation of archaeological objects. This work is interested because there was a concern in the development of visual transitions between the visualization modes on each display, i.e., the graphics became 3D from 2D projection on the screen.

\citet{BaumgartnerED07} presented an HUI to explore the organization of a desktop, disposing documents spatially. The organization of documents is done through gestures, using a glove. The documents in the 3D space are visualized in an autostereoscopic display and edited with the keyboard and a pen on a tablet below the display.

%Carvalho~\cite{Carvalho07} developed an HUI with the goal of exploring different 3D interaction tasks in a single work environment. VR was applied using an HMD for navigation tasks, and AR was used for manipulation tasks. WIMP (Windows, Icons, Menu and Pointer) interfaces were used for common desktop tasks, such as text editing.

In general, the above-mentioned works sought to blend different technologies in order to execute certain tasks. However, there is the need for a criterion, or at least a reference to any methodology or concept, driving this ``technological mixture''. Moreover, there seems to be no explicit concern about possible transitions among the integrated technologies.

Conversely, although the concept of transitional interfaces was coined a decade ago in the MagicBook project~\citep{Billinghurst01}, until now few works have been published in this topic. \citet{GrassetLB06} proposed an initial attempt to formalize this concept, and also published the first work ~\citep{GrassetDB08} reporting some evaluation studies in this field. Usability, performance, presence and awareness were assessed in that work. The evaluation scenario was composed by a series of navigation tasks that forced the subjects to transit between two interactive environments: virtual reality and augmented reality. However, most of the results from this work reported failures on navigation techniques used in each environment, as well as problems of disorientation on the entry and exit points from the environments after and before the transitions. The authors said that such problems would be fixed by using stereo vision and some sort of visual feedbacks.

The design and evaluation of transitional interfaces is still an open problem. There are lots of issues to be addressed, for instance, the problem of transition between interactive environments with different hardware and software technologies. While Grasset's work reported results on a scenario with a homogeneous hardware setup (HMD and a 3D input device) along the two environments, our work deals with explicit hardware and software transition, taking into account some properties to guide the design and evaluation of such case.

%------------------------------------------------------------------------- 
\section{Transition design based on the continuity of interaction}
\label{design}
In this section we briefly introduce the definition of continuous interaction based on the continuity properties: perceptive, cognitive and functional, followed by the proposed design approach in order to reach smooth transitions in HUIs.

\subsection{What's interaction continuity?}
\label{sec:continuity}
An interactive environment is assumed to be the complete presentation environment required for carrying out a particular interactive task. The interactive environment contains representations of the visual, haptic and auditory elements that a user interface offers to its users, as well as their relationships. In more complex environments the user`s task can be distributed along various interactive environments and discontinuity during transitions can become a problem. The idea of continuity is related to the process of avoiding breaks during interaction, which could result in disorientation and failure. 

The importance of continuity in the development of real-time collaboration systems was mentioned by \citet{Ishii94}. They define a seam as a spatial, temporal or functional constraint that forces the user to shift among a variety of spaces or modes of operation. For example the seam between word processing using a computer and traditional pen and paper makes it difficult to produce digital copies of handwritten documents without a translation step. All authors have agreed that systems asking users to abandon their acquired skills and to learn a new protocol are likely to encounter strong resistance. 

\citet{DuboisCont} consider continuity at the perceptual and cognitive levels. Perceptual continuity is present if the user perceives the different representations of a given entity directly and smoothly. Cognitive continuity is present if the cognitive processes that are involved in the interpretation of the different perceived representations are similar.

Here we consider continuity as the capability of the system to promote a smooth interaction with the user during task accomplishment considering perceptual, cognitive and functional aspects ~\citep{TrevisanCHI, TrevisanVRJ}. Perceptual continuity is defined as the ability of the system to provide information to the user in one perceptual environment (e.g. when the user is wearing a see-through head-mounted display (HMD)). Cognitive continuity is defined as the ability of the system to ensure that the user will interpret the perceived information correctly and that the perceived information is correct with regard to the internal state of the system (e.g. by using similar representations of the real and virtual objects). Functional continuity is defined as the adaptability of the user to change or learn new modes of interaction. Consequently the functional property is related to the interaction technique used.

All these definitions pointed out the need for smooth transitions between different operation modes to avoid frustration during the accomplishment of complex tasks. 

\subsection{Design approach}
\label {design_approach}

When there are multiple sources of information and two or more interactive environments we must make choices about what to attend to and when. At times, we need to focus our attention exclusively on a single item without interference from other items. At other times, we may need to time-share or divide our attention between two (or more) items of interest, which can be part of the same or different interactive environments.

For example, assuming real and virtual worlds as two different environments, in the Museum project ~\citep{RekiNagao95} the user wears a see-through head-mounted display in which information about an exhibit is displayed. The user is thus able to perceive real objects (the exhibit) and added synthetic information. The object of the task here is the painting in the exhibit. Therefore, the task focus belongs either to the virtual world or to the real world. 

The user could be either performing a task in order to manipulate or modify an object in one or more interactive environments.
By considering all the possibilities of interaction focus while the user is performing a specific task, we have identified three possible combinations:

\begin{itemize}
	\item Interaction focus in one interactive environment without shared attention. In this type of interaction attention is focused on a single object in the environment. There is no other object competing for the user's attention.
	\item Interaction focus shared in the same interactive environment. In this case the interaction focus is shared between two or more objects of the same environment.
\item Interaction focus shared between interactive environments. Here the interaction focus is shared between objects belonging to different environments.
\end{itemize}

Regarding the last item where the interaction focus can be shared between different environments, we must define how will be the insertion context of the interactive environments. Here we are interested to define the space around us according to the user's focus while performing a task. An interactive environment can be concretized using any device (screen, HMD, etc.) or any physical object (projected at a table, wall, etc.). Many disciplines can provide appropriate figures for physical properties. For instance, perceptual psychology may informs us to what extent a person considers an object far or close. Sociology interprets this as four spatial zones depending on the distance from subject. 

Here we are following the four spatial zones (see Fig.~\ref{fig:zones}) stated by \cite{Hall1969} for describing interactive environment, depending on the level of periphery:

\begin{itemize}
	\item Central zone. Corresponds to an insertion distance of 0 to 45 cm from the user.
	\item Personal zone. Corresponds to an insertion distance of 46 cm to 1.2 m from the user.
	\item Social zone. Corresponds to an insertion distance of 1.3 to 3.6 m from the user.
	\item Public zone. Corresponds to an insertion distance greater than 3.6 m from the user.
\end{itemize}

\begin{figure}[hbt]
\centering
\includegraphics[width=0.90\textwidth]{zones.eps}
\caption{Insertion Zones according to level of periphery.}
\label{fig:zones}
\end{figure}

If the interactive environment is inserted in the central zone of the user's task, s/he does not need change her/his attention focus to perform the task. If the user's attention focus is changing all the time, then it is probable that the interactive environment has been inserted outside the central zone, in a peripheral context of use. 

Taking into account our design approach for the proposed HybridDesk System it is assuming transitions between two zones of displays insertion: personal and central. Conforming is described in the next section~\ref{sec:HybridDesk-implementation}, the navigation environment is placed on the personal insertion zone while the manipulation and WIMP environments are placed on the central insertion zone.

In the HUIs/Transition approach, we suggest, as a guideline, that the design should first analyze the needs of the main task that the user intends to address in terms of interface. At this stage, it is worth thinking about the need of interfaces with different paradigms, such as Virtual Reality, Augmented Reality, WIMP (Windows, Icons, Menu, and Pointer), etc., and whether it is useful to have a great cognitive effort in the transition between the different interfaces. 

Once the interfaces are defined, the next step focuses on the definition of input devices, displays, and interaction techniques for each interface. This definition involves the analysis of perceptual and functional properties for transitions. The more different the devices and displays of each interface are, the greater the discontinuity between them will be. This decision should take into account a tradeoff between the advantage of the different interactive environments and the effects of discontinuities caused by transitions between them. For example, the imposition of a technology to avoid a discontinuity during a transition may result in poor performance of a task in an interactive environment. On the other hand, the use of different interactive environments that demand frequent transitions with any kind of discontinuity may be exhausting, and we may need to consider the use of some features during a transition to smooth any discontinuities.

The notion of interplay among the three continuity properties is important to handle transition discontinuities. As we show in the present work, the enhancement of a continuity property against a discontinuity of another one may contribute to smooth the transition between interactive environments.


\section{Conception of the HybridDesk System}
\label{sec:HybridDesk-implementation}

The design of the HybridDesk (Fig. ~\ref{fig:esquema}) was based on the requirements found for a 3D annotation task in an oil and gas application ~\citep{Carvalho-VRCAI}. These requirements helped to identify the technological setup used to compose the interactive environments within HybridDesk. This setup should be able to address the requirements of a semi-immersive workplace with stereoscopy, headtracking and a wand as well as the requirements of a WIMP interface.\footnote{A video from the system and user interactions is available as suplementary material and at the web address: http://www.youtube.com/watch?v=Gd4jRlIKJSM.} 

\begin{figure}[hbt]
\centering
%\includegraphics[bb=0 -25 85mm 55mm]{HybridDesk_Esquema_Envs.png}
%\includegraphics[width=0.90\textwidth]{HybridDesk_Esquema_Envs_new01.png}
\includegraphics[width=0.90\textwidth]{HybridDesk_Esquema_Envs_new01.eps}
\caption{HybridDesk scheme with the three interactive environments.}
\label{fig:esquema}
\end{figure}

\subsection{3D Annotation Task}
\label{3D_Annotation_Task}
The 3D annotation task was considered important because it could be decomposed into different subtasks ranging from symbolic input information to spatial input used for 3D interaction techniques. The 3D annotation task is a way to insert information into a virtual environment, and often this information is related to a particular object. To handle the creation and management of symbolic information we have chosen to take advantage of past experiences of users with WIMP interfaces rather than creating a completely new environment. Basically, we want to use all the resources available in a WIMP interface to create files, and “3D shortcuts” (3D Icons) to these files are created inside the virtual environment by the user. The visual feedback of the shortcuts has a shape of a 3D cube (Fig. ~\ref{fig:annotation}).

\begin{figure}[hbt]
\centering
\includegraphics[width=0.45\textwidth]{428_fig_tese.eps}
\includegraphics[width=0.5\textwidth]{424_fig_tese.eps}
\caption{Attaching the 3D annotation into a virtual environment. Left: Selecting the annotation. Right: Attaching an annotation icon.}
\label{fig:annotation}
\end{figure}

Annotations are very important in oil and gas applications ~\cite{RAPOSO-2009}. For instance, a virtual oil platform is composed by a huge number of different objects with many professionals working on it, and the need to leave different information (audio data, text data, presentations,movies, etc.) attached to some objects is important to convey some intentions or observations.

Regarding 3D interaction aspects related to the 3D annotation task, we have identified three subtasks:

\textbf{1) 3D Navigation and 3D Selection :} to search and select an object or to explore the 3D virtual environment. Models of oil platforms are extremely dense geometric scenarios, with plenty of things to explore. For these subtasks, Virtual Reality and some associated 3DUI techniques are appropriated interface paradigms.

\textbf{2) 3D Manipulation :} to manipulate objects through translations and rotations, and to enable the insertion of annotation using 3D Icons. Virtual Reality is also a suitable interface paradigm for this subtask.

\textbf{3) Symbolic Input :} to create and edit textual annotations. To handle the creation and management of symbolic information we have chosen to take advantage of user's past experiences with WIMP interfaces rather than to create a completely new environment.

\subsection{Interactive Environments}
\label{Interactive_Environments}

To support the subtask requirements and the appropriate interface paradigms described above, we designed three interactive environments for the HybridDesk:

\begin{figure}[h]
\centering
\includegraphics[width=0.75\textwidth]{ws_vr_nav_new.eps}
\caption{VR-Nav interactive environment.}
\label{fig:VR-Nav-Env}
\end{figure}

\textbf{1) VR-Nav:} some studies have indicated that navigation tasks tend to benefit from displays with larger FOV (Field of View) and FOR (Field of Regard)  ~\citep{Raymaekers05,Czerwinski02,Tan06}. Because of the larger visual space of these displays, their properties also provide better support for searches during selection tasks ~\citep{SteedParker05,RajaBowman06}. The hardware setup of VR-Nav is composed by four projection screens that are used simultaneously to provide a broader FOV and a larger physical visualization area near the user. The display resulting from this set of screens was named MiniCave and was inspired by the idea of CAVE adapted to a desk (Fig. ~\ref{fig:VR-Nav-Env}). A wand (a Wiimote tracked by an optical tracker) is used as input device, and a 3D representation for it was created in the virtual environment. This representation was visually similar to the real device and receives the translations and rotations from the tracker device. The raycasting technique was implemented for selection and ``grabbing in the air'' technique~\citep{Bowman05} for navigation task. According to the level of periphery of the displays insertion introduced in Section~\ref{design_approach} this environment is placed on the personal zone of user interaction.

\begin{figure}[h]
\centering
\includegraphics[width=0.75\textwidth]{ws_vr_manip_c_legenda.eps}
\caption{VR-Manip interactive environment.}
\label{fig:VR-Manip-Env}
\end{figure}

\textbf{2) VR-Manip:} for manipulation tasks, we have understood that it is important to perform a detailed visual inspection of a selected object in order to identify particular points to leave annotations. We therefore chose an isomorphic interaction technique\footnote{\textit{The isomorphic Interaction Technique} suggests ``\textit{a strict, geometrical, one-to-one correspondence between hand motions in the physical and virtual worlds...}''~\citep{Bowman05}}. to handle virtual objects at a short distance, like people do with real objects. Local Manipulation tasks have shown evidence of being more prone to visual stimuli in a restricted visualization area, which suggests the need for a narrower FOV ~\citep{SteedParker05,DemiralpKZL06,PrabhatFKWS08}. However, as the focus of attention is more intense in this restricted area, visual cues are more noticeable. Thus, display features such as brightness, resolution, sharpness, and photorealism become important~\citep{Nicholas05}. The inspiration for the technological setup came from the idea of a Reachin display which allows the manipulation of virtual objects within the arm's reachable space. Most Reachin displays use a mirror or a semi-transparent surface to show an illusion of a virtual content behind it. Such illusion is based on the reflection of an image from a CRT or a projection screen. For all these reasons, we have choosen an LCD to help during the manipulation task (Fig. ~\ref{fig:VR-Manip-Env}). Although a regular LCD is much thicker than a mirror or glass, we believe it provides reasonable access to the space behind it, which is the area of interaction with the virtual objects selected on the VR-Nav. Moreover, a regular LCD has all required display features to facilitate the visual inspections. According to the level of periphery of the displays insertion introduced in Section~\ref{design_approach} this environment is placed on the central zone of user interaction.

\textbf{3) WIMP:} basically, we desire to use all the resources available in a WIMP interface to create files, and ``3D shortcuts'' (3D Icons) to these files are attached to the associated 3D objects in the VR-Manip environment by the user. The visual feedback of the shortcuts has the shape of a 3D cube. As WIMP interface, we choose Microsoft Windows XP, and, concerning its associated hardware, we maintained the conventional setup, composed by monitor, keyboard, and mouse.  According to the level of periphery of the displays insertion introduced in Section~\ref{design_approach} this environment is placed on the central zone of user interaction.

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{Transicoes_eventos.eps}
\caption{Trigger Events.}
\label{fig:ws_trans_eventos}
\end{figure}

Fig. ~\ref{fig:ws_trans_eventos} shows a scheme of the events that trigger the transitions in the HybridDesk. The transition VR-Nav\textgreater VR-Manip is triggered by pointing to an object in the virtual environment (using the ray casting technique) and clicking on the button ``A'' of the wand. After that, the user has to move the LCD to the front of the MiniCave. To return from VR-Manip to VR-Nav, it is necessary to click on the button ``Home'' of the wand. The user then has to move the LCD out of the MiniCave.


The transition VR-Nav\textgreater WIMP is triggered by pointing to some 3D icon attached in some virtual object and clicking on the button ``A'' of the wand. To return to the VR-Nav it is necessary to use a context menu and select ``Back to HybridDesk VR-Nav''. 

The transition from VR-Manip to WIMP is triggered by clicking on the button ``A'' of the wand, and from the WIMP to VR-Manip, the user needs to use a context menu over the selected file and select ``Send to HybridDesk VR-Manip''. 

%To attach a 3D Icon at the tip of wand to a 3D object it is triggered by clicking on the button ``A''
%To choose a file inside WIMP and attach to a 3D object is required to go to VR-Manip first and then trigger the transition to WIMP by clicking on the button ``A'' of the wand. To send a file to VR-Manip, the user needs to trigger a context menu over the selected file and click on the submenu ``Send to HybridDesk VR-Manip''. After this, in VR-Manip a 3D icon appears at the tip of wand relating to the previous selected file in the WIMP.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% TRACK : INCLUSAO NOVO %%%%%

\subsection{Technology Setup}

\begin{figure}[hbt]
\centering
\includegraphics[width=0.8\textwidth]{ws_triplehead2go_c_legenda.eps}
\caption{(a) Two Matrox TripleHead2Go splitters to extend (b) the desktop through all the screens (c) using only one graphics board with two video outputs.}
\label{fig:triplehead2go}
\end{figure}

Instead of using a cluster of PCs and synchronization mechanisms, the semi-immersive visualization at the HybridDesk was managed by a single PC with a 3D accelerated video card with two video outputs. The HybridDesk needs a minimum of five video outputs:  4 to projectors and one to the \textit{LCD}. To handle this configuration we used two video \textit{splitters}. These \textit{splitters} are of the type \textit{TripleHead2Go} (Figure ~\ref{fig:triplehead2go}.a) and they divide one video output in three others. Basically each \textit{splitter} has a driver which increases the resolution of the desktop over a video output, and then the hardware splits it.  For example, a video output that would normally have a resolution of 1024x768 pixels, using this driver it can be transformed into an output with a resolution of 3078x768 pixels and then be divided in three outputs of 1024x768.  Thus, an extended desktop of 6 video output was created, three of 800x600 (2400x600) and the three of 1024x768 (3072x768) (Figure ~\ref{fig:triplehead2go}.c). These outputs together formed a large desktop display over the projectors and the LCD (Figure ~\ref{fig:triplehead2go}.b). 

\begin{figure}[hbt]
\centering
\includegraphics[width=0.65\textwidth]{ws_tracker.eps}
\caption{Optical Tracker composed by two infrared cameras.}
\label{fig:ws_bratracker}
\end{figure}

The three-dimensional effect was achieved using the stereoscopic anaglyph in simple projection screens. As the generation of stereoscopic image was all done via software, and only one resulting image was displayed for each screen, there was no need to use special screens neither the use of advanced projectors. The video card used was NVIDIA GeForce 8800 GT, a conventional accelerator board.

\begin{figure}[hbt]
\centering
\includegraphics[width=0.65\textwidth]{ws_oculos_wand_c_legenda.eps}
\caption{(a) Anaglyphic Stereo Glasses and (b) Wiimote used as a Wand.}
\label{fig:ws_oculos_wand}
\end{figure}

Using a Wiimote (Figure~\ref{fig:ws_oculos_wand}.b) as a Wand was a reasonable choice, since it is cheap and offers a variety of events without the need for wires, i.e, all data communication is done via bluetooth. The system also used a optical tracking system called Bratrack (www.bratrack.com) to estimate the translations and rotations needed for the navigation and manipulation tasks and headtracking (Figure~\ref{fig:ws_bratracker}). Track points were attached in the wiimote as well in the stereoscopic glasses (Figure~\ref{fig:ws_oculos_wand}).

\begin{figure}[hbt]
\centering
\includegraphics[width=0.65\textwidth]{ws_3d_views_c_legenda.eps}
\caption{(a)(b)Set of virtual cameras used in MiniCave. (c) Extra camera to provide an overview of the user position inside virtual world.}
\label{fig:ws_3d_views}
\end{figure}

To generate and manage 3D graphics, a system was implemented using the \textit{OpenSceneGraph} library (www.openscenegraph.org), which is based on the idea of scene graphs. The system rendered and managed six virtual cameras: 4 for the MiniCave (Figures ~\ref{fig:ws_3d_views}.a and ~\ref{fig:ws_3d_views}.b), one for the LCD , and an extra camera for an overview of the position of the observer within the virtual environment (Figures ~\ref{fig:ws_3d_views}.c and ~\ref{fig:desktop_extendido}.h). In order to create the anaglyph stereoscopic images, 10 images per frame were computed to generate the left and right eyes for each virtual camera (the extra camera did not use stereoscopy).

\begin{figure*}
\centering
\includegraphics[width=1.0\textwidth]{screenshot_desktop_extendido_c_legenda.eps}
\caption{Screenshot of the extended desktop showing the 6 windows for the screens: (a) left, (b) front, (c) right, (f) LCD, (g) inferior, amd (h) overview. In (d) and (e) are empty spaces generated by the splitter driver to compose a rectangular final image for the screenshot.}
\label{fig:desktop_extendido}
\end{figure*}

For each virtual camera, the system also created a window on the desktop so that each window occupied an area corresponding to a video output. In Figure ~\ref{fig:desktop_extendido} there is a screenshot showing the whole desktop. 

The MiniCave screens were designed using pieces of glass for the three vertical screens and a piece of acrylic for the table top. Projection screens were placed inside the Minicave over the surfaces of the glasses and the acrylic.
A set of mirrors was used to decrease the distance between each projector and the corresponding screens (Figure ~\ref{fig:espelhos}). For this, wooden frames containing a mirror have been constructed (Figure ~\ref{fig:espelhos}.a). Keystone correction was used on some projectors due to the projector's position and orientation adjustments, for instance, the inclination of the projector on Figure ~\ref{fig:espelhos}.c.

\begin{figure}[hb]
\centering
\includegraphics[width=1.0\textwidth]{espelhos_c_legenda2.eps}
\caption{(a)Mirrors in MiniCave. Pair Projector/Mirror on screens: (b)frontal, (c)under the table and (d)sides. (e) Overview of the HybridDesk structure during its development}
\label{fig:espelhos}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Conception of transitional interfaces}
\label{sec:Transitions-conception}

Once the user's interaction focus during the execution of the 3D annotation task is shared between objects belonging to three different environments, this section describes our design choices based on the previous interactive environments to analyze the continuity properties (see Section~\ref{design}) and to implement features to smooth identified discontinuities in transitions.

In Fig. ~\ref{fig:ws_continuidadeHD}, there is a scheme of devices and displays used in each interactive environment as well as the continuity issues during the transitions between them. Care was taken in these transitions to maintain at least one continuity property between the environments. In the VR-Nav to VR-Manip transitions and vice-versa, we maintained cognitive and functional continuity, since the representation of objects are the same as well as the input device. In the transitions both from VR-Nav and VR-Manip to WIMP, there is a functional discontinuity, since the user changes the input devices, from the wand to mouse/keyboard. In these transitions, we maintained cognitive continuity by associating two similar representations (2D Icon in a WIMP file manager and 3D Icon in virtual environment) of the same concept (data file). 

\begin{figure*}
\centering
\includegraphics[width=0.8\textwidth]{transitions-continuity_en.eps}
\caption{Continuity properties for the HybridDesk prototype.}
\label{fig:ws_continuidadeHD}
\end{figure*}


\begin{figure}[hbt]
\centering
\includegraphics[width=0.75\textwidth]{Transicao_devices.eps}
\caption{Devices used in each environment and the necessary changes during the transitions.}
\label{fig:ws_trans_devices}
\end{figure}

In Fig. ~\ref{fig:ws_trans_devices}, we illustrate the devices and displays used in each interactive environment, as well as the changes of input devices (functional discontinuities) and displays (perceptual discontinuities) required during transitions.

There is an important event during the transition between VR-Nav and VR-Manip that is related to a display change (Fig. ~\ref{fig:teaser:01}.c and ~\ref{fig:teaser:01}.d). Looking into the continuity properties, such event could be classified as a perceptual discontinuity. We can expected a potential perceptual discontinuity between VR-Nav and WIMP because these environments use distinct displays, MiniCave and LCD,  which are inserted in different insertion zones of user interaction (see Section~\ref{design_approach}): personal and central zones respectively. On the other hand a potential continuity could be expected from VR-Manip and WIMP transitions once booth interactive environments are using the same display (LCD) placed on the same central insertion zone.

Since we opted to use different display that brings advantages for each subtask (manipulation and navigation), we decided to handle such perceptual discontinuity by exploring the cognitive and functional properties. For the cognitive property, we have implemented an animation mechanism as a visual sign to guide the user during the transition, showing what is happening and what to do next. This is an attempt to visually explain the transition from one display to another. For instance, when the user selects an object in VR-Nav, an animation starts to move the virtual camera closer to the selected object. During this approximation, another animation (a virtual LCD moving from left to right and a label showing the name of the incoming environment - VR-Manip) appears, with a sign to move the LCD to the front of the MiniCave (Fig. ~\ref{fig:teaser:01}.a). The transition from VR-Manip back to VR-Nav shows an animation moving the camera back to the location where the object was selected before entering VR-Manip, and, at the same time, another animation shows a virtual LCD moving from right to left indicating the change of displays again (Fig. ~\ref{fig:teaser:01}.b). For the functional continuity property, we also choose interactive techniques with certain similarities in both environments, since VR-Nav and VR-Manip are 3D interactive environments and share the same input device (wand). The technique for locomotion in VR-Nav and the one used in VR-Manip are almost identical because the commands and spatial movements to move the camera in VR-Nav are the same ones to move the selected object behind the LCD in VR-Manip. 


\begin{figure}
\centering
\includegraphics[width=0.45\textwidth]{teaser_fig04.eps}
%\label{fig:teaser:05}
\includegraphics[width=0.45\textwidth]{teaser_fig06.eps}
%\label{fig:teaser:06}
\includegraphics[width=0.45\textwidth]{teaser_fig07.eps}
%\label{fig:teaser:07}
\includegraphics[width=0.45\textwidth]{teaser_fig08.eps}
%\label{fig:teaser:08}
\caption{(a) and (b) Visual signs used during the transitions between the VR-Nav and the VR-Manip environments, (c) and (d) Change of devices and displays during transitions.}
\label{fig:teaser:01}
\end{figure}

The transitions between the 3D environments (VR-Nav and VR-Manip) and the WIMP interface could be considered abrupt because there is a change in the interaction paradigm, from 3D to 2D. One of the main reasons for such abrupt change is the presence of functional discontinuities along the transitions. These discontinuities occur due to both the change of input devices (mouse/keyboard - wand)(Fig. ~\ref{fig:teaser:01}.c and ~\ref{fig:teaser:01}.d) and the difference in the nature of the environments in which the devices are used. The continuity we established during the transitions from WIMP to other environments was based on cognition, using interpretation and inferences. For instance, when the user arrives in the WIMP environment from VR-Manip and chooses a file inside the file manager, and opens a context menu, and the command to send the file to VR-Manip is activated (Fig. ~\ref{fig:ws_trans_eventos}), then VR-Manip reappears, but with a 3D Icon. We believe this visual difference in VR-Manip before and after using the WIMP interface (without the 3D Icon before, and with the 3D Icon after using it) helps the user to infer that this 3D Icon is a shortcut to the selected file in the previous environment. When the user perform the transition from VR-Nav to WIMP by pointing and clicking on a 3D Icon attached to an object in the virtual environment, the WIMP interface appears on the LCD with an open file manager and a selected file. We also believe that such a transition suggests an interpretative link between the environments (cognitive property), i.e. something selected in VR-Nav followed by the appearance of something selected in WIMP can be interpreted as the same object (two representations of the same concept).

The following section verifies, through an exploratory study, whether such design choices were properly perceived by users.

\section{Exploratory study}

This section aims to report an exploratory study on the interaction and behavior of some subjects during the transitions between the three environments designed for the HybridDesk: VR-Nav, VR-Manip, and WIMP. 

The methodology considers the study of the interactions during transitions in accordance with the perceptual, cognitive and functional continuity properties. It also provides data on the effectiveness of the system according to the subjects.

As we mentioned previously, the HybridDesk was developed following the requirements of a 3D annotation task. In order to validate this task, an application prototype consisting of a virtual oil and gas platform was implemented. Thus, the task of annotation might be interpreted, for example, as part of a collaborative task management of CAD modeling of an oil and gas platform.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% TRACK : INCLUSAO NOVO %%%%%



The goal of this test was an usability evaluation of transitions and tasks and it was conducted using the talk aloud approach. Also, an  analysis of the continuity properties in transitions/tasks was made to evaluate the design decisions.

\subsection{Subjects and Tasks}

This test was conducted with six subjects and all of them had some type of previous experience with 3D applications. A talk aloud approach was used and instead of giving an oral explanation of the application scenario (oil and gas virtual scenario), the following text was given:

\textit{``You are an experienced user in 3D model graphic visualization, but never used the HybridDesk before. 
You were hired by an oil company to verify the modeling quality of 3D objects developed by another group within the company.
You will use the HybridDesk on a daily basis to perform the verifications and leave annotations, with your assessment, attached to the verified object.
Your manager has left instructions regarding your task in the 3D scene, in an annotation file attached to the top of the stair at the right side of the heliport of the oil platform.
You must navigate at the virtual oil platform using the HybridDesk, find the annotation file and follow the instructions left by your manager in this file.''}

The annotation file was a shortcut to a text file located in the file manager. Inside this text file there was the following instruction:

\textit{``Go to VR-Nav, find a crane which contains a fissure on its tip, and create an annotation near the fissure to indicate this problem''} 

A training phase was conducted during 10 minutes for each subject. 

\subsection{User Test - Results of the User Observation with Talk aloud}

In order to estimate how was the continuity during the transition interactions, the evaluation issues were classified into three categories following the continuity properties: cognitive, perceptual and functional (see Sections~\ref{design} and 
~\ref{sec:Transitions-conception}). The average time of user's sessions took 11 minutes.



\begin{table} [hbt]
% table caption is above the table
\caption{Issues during User Observation Sessions on Test 2 and their classification based on Continuity Properties.}
\label{tab:evaluation_table2}       % Give a unique label
% For LaTeX tables use
\begin{tabular}{llll}
\hline\noalign{}
\textbf{Perceptual Property - Transition} &  \textbf{Observations}\\
\noalign{}\hline\noalign{}
1. Tried to interact with manipulation as if it was in navigation & 2 \\
2. Returned to WIMP without noticing what was going on & 1 \\
\hline\noalign{}
\textbf{Perceptual Property - Task} \\
\noalign{}\hline\noalign{}
3. Turned around the whole 3D scene to verify the tip of the crane & 4 \\
4. Created a new file because could not find the\\
previously created file & 1 \\
5. Did not notice if the 3D icon was attached to the object or not & 1 \\
6. Selected again the annotation icon to verify the task was\\
correctly done & 1 \\
7. Was in doubt if the annotation was attached or not, and reselected & 1 \\
\hline\noalign{}
\textbf{Cognition Property - Transition} \\
\noalign{}\hline\noalign{}
8. Went to WIMP intentionally, but wanted navigation & 1 \\
9. Tried to return using wand commands & 1 \\
10. Tried to return using the back option & 1 \\
11. Was lost, not knowing how to return to navigation & 1 \\
\hline\noalign{}
\textbf{Cognition Property - Task} \\
\noalign{}\hline\noalign{}
12. Lost orientation when navigated through the crane model & 1 \\
13. In doubt of which were the left and right sides of the Heliport & 1 \\
14. Went initially to the wrong stair & 3 \\
15. Was initially disoriented when returned to the original position & 1 \\
16. Thought that it would be difficult to identify the crane & 1 \\
17. Selected the tip of the tower instead of the tip of the crane & 1 \\
18. Selected the hook thinking it was the tip of the crane & 1 \\
19. Selected again the annotation icon because had forgotten the task & 1 \\
\hline\noalign{}
\textbf{Functional Property - Transition} \\
\noalign{}\hline\noalign{}
20. Pressed the A button unintentionally when disrupted by\\
the LCD (left-handed) & 1 \\
21. Went to WIMP unintentionally, due to a slip int A button & 1 \\
22. Pressed the A button twice and returned to WIMP & 1 \\
23. Went to manipulation unintentionally, due to a slip int A button & 1 \\
24. Pressed the A button too long and went straight to WIMP & 3 \\
\hline\noalign{}
\textbf{Functional Property - Task} \\
\noalign{}\hline\noalign{}
25. Difficulty to approximate to the platform when far away & 4 \\
26. Selected the 3D icon correctly but with difficulty & 1 \\
27. Terminated the session without verifying the second crane & 1 \\
28. Tried to delete the attached annotation file but had no success & 1 \\
29. Selected an object instead of the annotation icon & 2 \\
30. Selected an object unintentionally & 2 \\
31. System was stuck in WIMP because name of the file was changed & 1 \\
\hline\noalign{}
\textbf{Ergonomics} \\
\noalign{}\hline\noalign{}
32. Found the glasses uncomfortable & 1 \\
33. Arm was tired & 1 \\
34. Preferred not to move the LCD to the center & 2 \\
35. Notepad message was visually obstructed by the LCD & 1 \\
36. Difficult to see the wand at the LCD due to the\\ 
left-handed position & 1 \\
\noalign{}\hline
\end{tabular}
\end{table}


As per Table~\ref{tab:evaluation_table2}, a large number of usability issues (36) were identified from the talk aloud session. By analyzing the nature of these usability issues we divided them into two categories: analysis of tasks into each environment and analysis of transition interaction. Then for each category we analyzed the continuity properties: cognitive, perceptual and functional (see Sections~\ref{design} and ~\ref{sec:Transitions-conception}).

Cognitive property of tasks are related to problems caused by signification, what was surprising, especially considering that in many instances the cause was the interpretation by the user of the meaning of the task, as defined by the task manager, represented by the evaluator in the test (items 12 to 19). Three out of the six participants (item 14) went initially to the wrong stair, because they did not understand the task correctly, which told them to look for the annotation at the ``...stair to the left of the Heliport.'' This kind of issue raises various questions, particularly what was the real cause of this misinterpretation, and how to avoid it. One difficulty is the need to translate instructions in a textual message to a 3D scene, where the stair had only a visual representation. The lack of orientation marks or maps in the 3D scene did not help either (items 12, 13 and 15). The participants also had some difficulty in interpreting what the actual meaning of ``tip of the crane'' in the instructions was, some confusing it with the tip of the crane tower, and others confusing it with the hook hanging from the crane (items 16, 17 and 18).


Functional property of tasks are related to execution problems while performing a specific task into a specific environment (items 25 to 31). One of the issues with greater execution incidence (item 25 - 4 out of 6 people) was the difficulty of traveling toward the 3D scene from a distant point of view. The user knew what to look for but did not know how to find his/her way in the 3D scene. This is why many of the issues identified, although they occurred in the navigation environment (VR-Nav). It was mentioned that when an object was far from the observer, the icon attached to it was too small to be perceived, and when the object was near the icon, it become too big. In conventional WIMP interfaces, the icons are always the same size when browsing through the folders of the system, unless the user intentionally changes the configuration options of the folder. This difference on icon sizes reduces even the cognitive continuity between WIMP and 3D environments. This is a scale problem, for which the designer could find a solution. Another difficulty was to make the selection of very small objects, in particular the 3D icon. Any small arm movement deviates the selection to another object. 

Perceptual property of tasks are related to problems indicating the need to improve system feedback (items 3 to 7). The lack of visual feedback when the 3D icon is attached to an object has also caused problems to some participants. 

The cognitive continuity analysis of transitions are mainly related to the memorization of commands (items 8 to 11). The ``Home'' button at the wand had also different meanings, depending on the context, which also caused confusion to some participants.

The perceptual continuity analysis of transitions are mainly related to the different insertion zones of the displays (items 1 and 2). 
In order to keep the user interaction focus on his/her task two perceptive environments were assumed. The environment located in the central zone was used to perform manipulation and textual tasks while the one placed on the personal zone was assumed to perform navigational tasks. Such design choice generated a perceptive discontinuity in order to support the most adequate environment for each user's task. Participants had difficulties to notice the result of their actions, which caused involuntary environment transitions that were not clearly perceived, although the system did provide visual animations as a feedback of transitions. One participant that did not notice the transition to the manipulation environment so he/she continued to interact with the system using the wand, as if he/she was still in the navigation environment. The fact that the navigation screens of the mini-CAVE remained apparently active when in the manipulation environment has misled this participant.

The functional continuity analysis of transitions are mainly related to the execution problems during transitions. Part of such problems were caused by unintentionally execution of wand commands (items 20 to 24) and  part of the execution problems were ergonomic (items 32 to 36), like fatigued arms, discomfort with the glasses and LCD manipulation. In the latter case, two participants chose not to slide the LCD towards the desk center for object manipulation, using the LCD at its left side position.  On the other hand, a left-handed participant was forced to move the LCD to the desk center, so that he/she could manipulate the object in the LCD using the wand at his/her left hand, but had some difficulties in moving the LCD.

Despite the fact of the user interaction took a short period of time during the test session was possible to notice few  ergonomics problems related to fatigue arm and discomfort while using the stereo glasses. We believe that such problems could be more evident over a long period of interaction with the HybridDesk. On the other hand problems related to memorization of commands can decrease with the use.



%\begin{table*}[ht]
%\centering
%\includegraphics[width=1.0\textwidth]{marcus_table2.eps}
%\caption{Issues during User Observation Sessions and their classification as Execution (E), Navigation (N), Signification (S), and Perception (P).}
%\label{fig:evaluation_table2}
%\end{table*}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\section{Conclusion}
\label{conclusions}

This work introduced a discussion towards the design of transitional interfaces and pointed out the importance of these interfaces for both hybrid user interfaces and a broader range of interfaces with distinct paradigms and technological setups.

We also discussed issues related to the interaction continuity (perceptual, cognitive and functional properties) that transitional interfaces should be able to address. A guideline was suggested with some steps during design phase using these continuity properties. This guideline aims to define interactive environments composed by an interface (of the mixed virtuality or mixed reality continuum), input devices, interaction techniques, and displays. Initially, the guideline proposed a task analysis to identify any possible sub-tasks with different characteristics, and, after that, the definition of interface paradigms for each sub-task considering cognitive properties during the transitions between them. The next step focused on the definition of input devices, displays, and interaction techniques for each interface. This definition involves the analysis of perceptual and functional properties for transitions. The more different the devices and displays of each interface are, the larger the discontinuity between them will be.

Following the guidelines, a semi-immersive system, called HybridDesk and composed of three interactive environments, was developed, and transitions between the environments were explored in a 3D annotation task in an oil and gas scenario application. 

The transitions between the environments of HybridDesk in the 3D annotation task were analyzed by means of an exploratory study with six subjects. We applied a talk aloud evaluation  to capture the subjects' reaction on the design decisions based on the continuity properties. The results pointed a major incidence of functional problems (7 occurrences), followed by cognitive (4 occurrences) and finally the perceptual ones (3 occurrences).

These occurrences brought up the evaluator's attention to the fact that the user interaction with the HybridDesk actually involved at least three distinct signification systems: ``the system'' (wand commanding), ``the content'' (3D scene) and ``the task'' (textual instructions). A different person produced each of these significations systems: the system designer, the 3D modeler and the task manager.  In the standard desktop applications, this interaction inconsistency is not so visible and critical.

Finally, we conclude that so far there is no theory to guide neither a systematic design of transitions between interfaces in forthcoming hybrid user interfaces, nor their treatment. Although our user test results are still preliminary, this work shows that carrying out a detailed analysis of the requirements of a task and relating then to continuity properties constitute a significant initial approach in the construction of a solid base for the design of transitional interfaces for hybrid user interfaces. It is important to remind that the evaluation performed in this study was focused on transition interactions and  such approach can be adopted by designers of any application involving transitions between contexts in order to minimize potential discontinuities still in design time. 

As future work, we plan to use a tracked mouse as a universal mouse-pointer to handle both 3D and 2D interaction, reducing thus the functional discontinuity in the transitions between VR-Nav/VR-Manip and WIMP. Also we plan to detail a formal modeling of transitional interfaces to encompass the requirements of hybrid user interfaces.


\begin{acknowledgements}
The authors would like to thank all volunteers users that took part in the experiments. Thanks also to Petrobras, the National Council for Scientific and Technological Development (CNPq) and FAPERJ for the financial support of this work.  
\end{acknowledgements}

 
%\bibliographystyle{abbrv}
%%use following if all content of bibtex file should be shown
%\nocite{*}
\bibliographystyle{spbasic}
%\bibliography{Tese}
\begin{thebibliography}{30}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{{#1}}
\providecommand{\urlprefix}{URL }
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{DOI~\discretionary{}{}{}#1}\else
  \providecommand{\doi}{DOI~\discretionary{}{}{}\begingroup
  \urlstyle{rm}\Url}\fi
\providecommand{\eprint}[2][]{\url{#2}}

\bibitem[{Baumgartner et~al(2007)Baumgartner, Ebert, and
  Deller}]{BaumgartnerED07}
Baumgartner S, Ebert A, Deller M (2007) Dimensional congruence for interactive
  visual data mining and knowledge discovery. In: EuroVis, Eurographics
  Association, pp 99--106

\bibitem[{Benko et~al(2005)Benko, Ishak, and Feiner}]{Benko05}
Benko H, Ishak E, Feiner S (2005) Cross-dimensional gestural interaction
  techniques for hybrid immersive environments. In: Virtual Reality 2005, IEEE,
  pp 209--216

\bibitem[{Billinghurst et~al(1999)Billinghurst, Kato, and
  Poupyrev}]{Billinghurst01}
Billinghurst M, Kato H, Poupyrev I (1999) The magicbook - moving seamlessly
  between reality and virtuality. In: IWAR 99, pp 35--44

\bibitem[{Bornik et~al(2006)Bornik, Beichel, Kruijff, Reitinger, and
  Schmalstieg}]{Bornik06}
Bornik A, Beichel R, Kruijff E, Reitinger B, Schmalstieg D (2006) A hybrid user
  interface for manipulation of volumetric medical data. In: IEEE Symposium on
  3D User Interfaces, IEEE

\bibitem[{Bowman et~al(2005)Bowman, Kruijff, LaViola, and Poupyrev}]{Bowman05}
Bowman D, Kruijff E, LaViola J, Poupyrev I (2005) 3D User Interfaces - Theory
  and Practice. Addison-Wesley

\bibitem[{Butz et~al(1999)Butz, Hollerer, and et~al.}]{Butz99}
Butz A, Hollerer T, et~al (1999) Enveloping users and computers in a
  collaborative 3d augmented reality. In: IWAR 99, pp 35--44

\bibitem[{Carvalho et~al(2009)Carvalho, Raposo, and Gattass}]{Carvalho-VRCAI}
Carvalho F, Raposo A, Gattass M (2009) A transitional interface between 2d/3d
  functional spaces in a desktop semi-immersive system. In: VRCAI 2009 – The
  8th International ACM Conference on Virtual-Reality Continuum and Its
  Applications in Industry, ACM

\bibitem[{Czerwinski et~al(2002)Czerwinski, Tan, and Robertson}]{Czerwinski02}
Czerwinski M, Tan DS, Robertson GG (2002) Women take a wider view. In: CHI '02:
  Proceedings of the SIGCHI conference on Human factors in computing systems,
  ACM, New York, NY, USA, pp 195--202

\bibitem[{Demiralp et~al(2006)Demiralp, Karelitz, Zhang, and
  Laidlaw}]{DemiralpKZL06}
Demiralp C, Karelitz DB, Zhang S, Laidlaw DH (2006) Cave and fishtank
  virtual-reality displays: A qualitative and quantitative comparison. IEEE
  Transactions on Visualization and Computer Graphics 12(3):323--330,
  member-Cullen D. Jackson

\bibitem[{Dubois et~al(2002)Dubois, Nigay, and Troccaz}]{DuboisCont}
Dubois E, Nigay L, Troccaz J (2002) Assessing continuity and compatibility in
  augmented reality systems. Journal of Universal Access in the Information
  Society 1(4):263--273

\bibitem[{Feiner and Shamash(1991)}]{Feiner91}
Feiner S, Shamash A (1991) Hybrid user interfaces: Breeding virtually bigger
  interfaces for physically smaller computers. In: ACM UIST 91, ACM, pp 9--17

\bibitem[{Grasset et~al(2006)Grasset, Looser, and Billinghurst}]{GrassetLB06}
Grasset R, Looser J, Billinghurst M (2006) Transitional interface: concept,
  issues and framework. In: ISMAR, IEEE, pp 231--232

\bibitem[{Grasset et~al(2008)Grasset, Duenser, and Billinghurst}]{GrassetDB08}
Grasset R, Duenser A, Billinghurst M (2008) Moving between contexts - a user
  evaluation of a transitional interface. In: ICAT 2008: 18th International
  Conference on Artificial Reality and Telexistence, Keio University, Yokohama,
  Japan

\bibitem[{Hall(1969)}]{Hall1969}
Hall ET (1969) The hidden dimension. Anchor Books

\bibitem[{Ishii et~al(1994)Ishii, Kobayashi, and Arita}]{Ishii94}
Ishii H, Kobayashi M, Arita K (1994) Iterative design of seamless collaboration
  media. Commun ACM 37(8):83--97

\bibitem[{Masso(2008)}]{Molina08}
Masso JPM (2008) A structured approach to the development of 3d user
  interfaces. PhD thesis, ALBACETE, ES, adviser-Pascual Gonzalez Lopez

\bibitem[{Milgram and Colquhoun(1999)}]{MC99}
Milgram P, Colquhoun HJ (1999) A taxonomy of real and virtual world display
  integration. Mixed Reality, Merging Real and Virtual Environments, Ohmshda
  and Springer-Verlag pp 5--30

\bibitem[{Nakashima et~al(2005)Nakashima, Machida, Kiyokawa, and
  Takamura}]{Nakashima05}
Nakashima K, Machida T, Kiyokawa K, Takamura H (2005) A 2d-3d integrated
  environment for cooperative work. In: Symposium on Virtual Reality Software
  and Technology, ACM, pp 16--22

\bibitem[{Polys et~al(2005)Polys, Kim, and Bowman}]{Nicholas05}
Polys NF, Kim S, Bowman DA (2005) Effects of information layout, screen size,
  and field of view on user performance in information-rich virtual
  environments. In: VRST '05: Proceedings of the ACM symposium on Virtual
  reality software and technology, ACM, New York, NY, USA, pp 46--55

\bibitem[{Prabhat et~al(2008)Prabhat, Forsberg, Katzourin, Wharton, and
  Slater}]{PrabhatFKWS08}
Prabhat, Forsberg A, Katzourin M, Wharton K, Slater M (2008) A comparative
  study of desktop, fishtank, and cave systems for the exploration of volume
  rendered confocal data sets. IEEE Transactions on Visualization and Computer
  Graphics 14(3):551--563

\bibitem[{Raja et~al(2004)Raja, Bowman, Lucas, and North}]{RajaBowman06}
Raja D, Bowman D, Lucas J, North C (2004) Exploring the benefits of immersion
  in abstract information visualization. In: 8th International Immersive
  Projection Technology Workshop

\bibitem[{Raposo et~al(2009)Raposo, Santos, Soares, Wagner, Corseuil, and
  Gattas}]{RAPOSO-2009}
Raposo A, Santos I, Soares L, Wagner G, Corseuil E, Gattas M (2009) Environ:
  Integrating {VR} and {CAD} in engineering projects. In: IEEE Computer
  Graphics \& Applications, IEEE, vol~29, pp 91--95

\bibitem[{Rascar et~al(1998)Rascar, Welch, Cutts, and et~al}]{Rascar98}
Rascar R, Welch G, Cutts M, et~al (1998) The office of the future: A unified
  approach to image-based modeling and spatially immersive displays. In:
  SIGGRAPH, ACM, pp 179--188

\bibitem[{Raymaekers et~al(2005)Raymaekers, Boeck, Weyer, and
  Coninx}]{Raymaekers05}
Raymaekers C, Boeck JD, Weyer TD, Coninx K (2005) The effect of display size on
  navigation in a virtual environment. In: Proceedings International Conference
  on Enactive Interfaces - Enactive '05

\bibitem[{Rekimoto and Nagao(1995)}]{RekiNagao95}
Rekimoto J, Nagao K (1995) The world through the computer: computer augmented
  interaction with real world environments. In: Proceedings of the ACM
  Symposium on User Interface Software and Technology, ACM, pp 29--36

\bibitem[{Rekimoto and Saitoh(1999)}]{RekiSaitoh99}
Rekimoto J, Saitoh M (1999) Augmented surfaces: a spatially continuous work
  space for hybrid computing environments. In: Proceedings of CHI'99, ACM, pp
  378--385

\bibitem[{Steed and Parker(2005)}]{SteedParker05}
Steed A, Parker C (2005) Evaluating effectiveness of interaction techniques
  across immersive virtual environmental systems. Presence: Teleoper Virtual
  Environ 14(5):511--527

\bibitem[{Tan et~al(2006)Tan, Gergle, Scupelli, and Pausch}]{Tan06}
Tan DS, Gergle D, Scupelli P, Pausch R (2006) Physically large displays improve
  performance on spatial tasks. ACM Trans Comput-Hum Interact 13(1):71--99

\bibitem[{Trevisan(2004)}]{TrevisanCHI}
Trevisan DG (2004) Designing smooth connections between worlds. In: Extended
  abstracts of the 2004 conference on Human factors and computing systems
  CHI2004, Session Doctoral Consortium, ACM Press, pp 1043--1044

\bibitem[{Trevisan et~al(2004)Trevisan, Vanderdonckt, and Macq}]{TrevisanVRJ}
Trevisan DG, Vanderdonckt J, Macq B (2004) Conceptualizing mixed spaces of
  interaction for designing continuous interaction. Virtual Reality Journal
  8(2):83--95

\end{thebibliography}

\end{document}

