Cell Phone Cost Calculator Usability Research Report

Hands-on Usability Testing

Cell Users

For the cell user groups, mock-up cell phone bills were created by Decima and Industry Canada so that the participants were working from the same base (see Appendix). With the first run through, time taken ranged from 2 to 12 minutes with most participants taking the full 10 minutes or a bit more and making it to the results screen.

Navigation

Participants had little trouble with the navigation. The layout and organization was considered by most to be well done. All the movement buttons (next, back, etc.) were clear and located where they assumed they should be. Other than some concern with the use of Suivi d'appel to reference call forwarding in French, there was a consensus that the language and directions were clear and straightforward. At the beginning, participants noted that they liked being able to choose between pre-paid and monthly plans (although few noticed how this affected the presentation of the results) and whether or not they make long distance calls.

One suggestion regarding the location of the buttons would be to place the "Submit my profile" button on the far right side. Participants got used to having the "Next" button there and they are more likely to do that than want to print.

Level of Effort

Participants were caught up most when having to break out their minutes. They understood why it was necessary to input that information, but it is still relatively difficult to itemize exactly how many minutes a person spends each hour of the day on incoming and outgoing calls.

Even though they were asked to consider their "personal usage," the few with cell phones used for business purposes commented that they never know where a client could be calling from so it's not easy to predict. This reaction regarding business usage is important to underline—if the tool is in fact specifically recommended for personal usage, more explicit language on the home page should be added to ensure there is no confusion among consumers who use their cell phone for business purposes.

Everyone agreed that the incoming minutes in general are very hard to predict. They feel they have very little control over that aspect of their call usage. As such, providing a typical usage number of 65% as a default ratio was considered a useful starting point by many.

Most agreed that inputting the usage by hour, splitting calls specifically by time of day, evenings and weekend, and incoming versus outgoing was taking the analysis too far. They felt this was quite frustrating, and expressed the added effort in needing a second calculator to figure out the numbers. Those who could in fact estimate their minute allocation felt the task was reasonable and did not require too much time to complete.

« Commencer à faire tout ces calculs c'est trop. »

« Pourcentage d'appels que je fais entrant ou sortant—j'ai aucune idée, je ne pense jamais en terme de pourcentage comme ça. »

Confusion also occurred between the outgoing and incoming long distance minutes break down pages because they looked almost identical. The page title was not clear enough, and they were forced to scroll down to get to the inputting fields. They would like to see the data fields closer to the top of the page without having to scroll down, or better yet, have all this information entered on the same screen (similar to the local minutes). While the local minutes only require one page of data entry, the inconvenience of scrolling down was noted here as well.

Many participants were bothered that the minute fields that couldn't be left blank needed to have a zero in them. They were brought to an error page if they left a field blank even though in their minds, a blank field is the equivalent to "zero". This confusion only arose when the participant deleted the default value of "zero" for some reason and did not enter an alternate value. Some also had trouble populating the cells pre-populated with zeros in that what was intended to be "1" ended up being "10" for instance. A programming recommendation might be that the field highlights when it is selected and the user simply overwrites the zero with their intended value.

Another comment regarding long distance minutes involved the allocation of minutes across time periods. Many did not notice that the allocation had to equal the total number of long distance minutes they had specified on the previous pages. Participants had two recommendations here: 1) that the total number of long distance minutes be incorporated upfront in the task description at the top of the page (e.g. "Distribute your 50 long distance minutes in the following time periods, then click on Next") and, 2) have auto deduction, i.e. an automatic countdown of the minutes as the user completes the fields.

Participants also felt their experience was too similar to completing a "spreadsheet" which they considered burdensome and awkward—they would have appreciated more of a "point and click" experience.

Also, if they wanted to change something and clicked 'back,' then they lost their previous information and they had to re-enter it. While this may be a function of the beta testing, it should be noted that participants do in fact expect and value that the website would save their data throughout their session.

Look and Feel

By and large, participants felt the layout was simple and visually appealing while only a limited few felt the site was not attractive. Besides a few minor aesthetic adjustments, participants were comfortable with the appearance of the site and the simple layout was appreciated.

There were mixed reactions to the 'Did you know' boxes on each page. Most participants did read them and for some it was new and valuable information. A good number of participants appreciated the relevancy of this information to the task at hand. There were a few who felt they did not add anything and they just didn't think these boxes should be so prominent. A key annoyance was that, visually, there was a lot of white space next to some of these boxes which forced the need to scroll down to get to the calculator input fields. A select few participants found it annoying to see this information first and then have to scroll to get to the input fields. They would rather see these boxes smaller, or more to the side or bottom of the page. It should be noted though that many participants did not read these boxes under the assumption that reading them was not part of the task. They were more preoccupied with completing their scenario task and opted to read this information on their own time.

Some of these observations are summarized on the following pages.

Output

Generally, participants were quite pleased with the results from the calculator. Although the results were fictitious, participants did like the presentation of the results and the data manipulation capabilities the calculator allowed. It should be noted though that most of the "sort" functions were not noticed because the visuals were not sufficiently obvious.

After just a few uses, participants had a lot less trouble getting through the calculator and it took less time. They also started to notice additional output features like being able to increase or decrease their usage by 20%, which they liked a lot—participants understood the usefulness of this feature in the event their behaviour changed. It should be noted though that many participants did not notice this feature on their own.

As well, it wasn't sufficiently clear that users could change their profile once they got to the end—a select few were frustrated having to re-enter all their numbers if they wanted to adjust their inputs a bit to see what happens.

Participants also liked the following aspects of the plan details page:

• They liked that actual costs are calculated as the "bottom line";
• They liked that all costs and fees are presented, even if they are not charged specifically to their profile, and,
• They liked the special notes section (e.g. additional discounts may be available, etc.).

Participants did note though that they would appreciate seeing some information on contracts and contract length incorporated into the plan details.

Finally, about half of the participants noticed they could compare up to three different plans. Most were pleased with this functionality—some would like to be able to compare up to five plans on the same page. As well, there was some interest in seeing more than 10 plans appear by default once they reach the results page.

Additional concerns from participants regarding various aspects of the tool included:

• It wasn't clear that the plans could be sorted. In fact, some did not even notice that the plans were sorted based on value by default. One participant saw the value in sorting by provider but questioned why he would want to sort results based on the plan name.
• The side-by-side results could be more visually appealing to help distinguish the three plans—perhaps the use of colors in the columns could assist in this endeavor.
• When specifically probed, participants noted that the tool does not take into account such things as: the cost of the phone, coverage (roaming), data usage (e.g. Internet surfing, music downloading, etc.), unlimited texting, GPS, taking and sending photos, activation fees and changing zone fees.
  • Participants did not mind that international calling was not included in the calculator. Most agree that there are better ways to call internationally than on cell phones.

Suggestions regarding various aspects of the tool included:

• Add information pages that provide additional research on things such as the difference between monthly and pre-paid plans.
• Include Information on coverage with different providers.
• All participants agreed there was value in including other types of plans such as family plans, network to network, etc. and they were prepared to input more information in order for the calculator to take these into account. A suggestion was to have an option at the beginning of the calculator (similar to how the long distance calls are handled) giving the user the option to include these plans in the analysis.
• A suggestion for the cumbersome detailing of minutes would be to use ranges of minutes as opposed to exact numbers.
• Additional information should be provided on the impact of signing a contract on monthly fees (i.e. participants were curious to know if the monthly fee is cheaper depending on the type of contract signed).
  
  

  « Ajoutez l'aspect des contrats—est ce que les frais notés sont disponibles avec (ou sans) contrat? Ou est ce que les prix sont reliés au contrat? »

• Some users simply wanted the ability to view plans and plan data without having to go through the profile builder. While this will be made available from the calculator's main page, it will need to be made more prominent.
• There was interest in being able to filter or funnel the presentation of plans based on cost or cost ranges.
• As well, when using the plan limit by provider, a few noted they would like to see an "All providers" option rather than having to select a blank to revert back to the list of all providers after having filtered.
• Finally, to appease their concerns regarding credibility and relevance, participants would like to see more explicit information on when the site was last updated.

Cell Intenders

Two different approaches were used to obtain feedback from cell intenders.

• Approach 1: Participants in Vancouver were asked to consider 3 lifestyle profiles and use the information in each profile to enter an anticipated usage profile into the cost calculator. This approach was used so that all participants were more or less considering the same information when creating their usage profile and to ensure that all modules were being tested. Participants used the existing online module for this exercise.
• Approach 2: Based on very strong feedback from the Vancouver sessions, the protocol was changed for sessions in Halifax and Montreal. Here, participants were asked to enter their own anticipated, personal usage profile using three different calculator layouts: the existing cell intender module, the cell owner module and a new, hybrid module created by the Industry Canada team soon after the Vancouver sessions were completed. For most sessions, because of the amount of time needed, participants were only able to use the existing cell intender module and the hybrid module.

Feedback has been grouped according to the two main flows used by cell intenders: the existing cell intender module, which was used in all cities, and the hybrid flow, which was only seen in Halifax and Montreal.

Existing Cell Intender Module

Overall Perceptions

The general idea of the calculator and its usefulness was grasped by the participants and they liked the concept. The calculator 'made sense' but many wondered how people were supposed to figure out what their anticipated usage was supposed to be. This concern was mostly regarding the detailed breakdown of the incoming and outgoing minutes pages. The participants got caught up and frustrated that they had to input so many details at the very beginning of the profile builder.

Navigation

On the whole, navigation was straightforward although there were some noteworthy issues. Some commented that the navigation buttons could be different sizes in terms of what one is more likely to press. For instance it was more likely that they would want to 'look at results' than 'print', so the print button should be smaller.

As well, on the pages where users are supposed to click "Calculate", not all users instinctively understood that this was needed and even among those who did click the right button, many did not realize that it showed them the same page again. Some participants even tried to click into the orange section of the grid to input numbers there, unbeknownst to them that these would auto-fill using the calculate function.

A related concern occurred when users clicked 'calculate' after all the inputting was complete. The screen goes to the top and they thought all the information had been lost. They would like to see it all calculated right there, with no change of screen position.

The typical usage patterns were by and large understood by most participants however questions were raised regarding the definition, and ultimately the applicability, of a "typical user" profile. Some wondered if it would be possible to create more narrowly predefined "typical user" profiles based on basic user information such as age, call behaviour, etc.

Finally, the appearance of the typical usage patterns was confusing to some. They did not know where to click to move on to the next screen since the navigation buttons, that are typically at the bottom of the screen, are now located between the user's entry grid and the typical usage pattern grid.

Level of Effort

The most frustration focused on the breakdown of minutes, especially for the evenings. Cell phone intenders generally did not understand why so much detail for evening calls was needed simply because they are less knowledgeable about how cell phone plans are structured. Not only were participants frustrated with the level of detail, but they were also confused by the similarity in the pages—many believed they were being asked to re-enter the same information or that they had made a mistake and lost all of their information when in fact they were on a new page. These navigation issues may have compounded frustrations for some of the users.

"The daily minutes, depends on the day, on the season, on the month or week."

"I like it, would like it better—say daytime—asking you how much you would use in a broader window of time. It'd be easier than figure out all the minutes in a single hour over the week. "

Given this confusion, there was interest in seeing the outgoing and incoming screens merged onto the same page.

Similar to cell phone users, new users also felt their experience was too similar to completing a "spreadsheet" which they considered burdensome and awkward—they would have appreciated more of a "point and click" experience.

Cell intenders also wanted to see more options provided at the beginning of the profile builder such as the option to specify: unlimited weekends, a certain number of daytime minutes per month and specific group plans. They would have liked to see more 'funneling' at the start when there were certain features that they knew they wanted. An extension of the funneling concept was interest in being able to pre-select or filter for a specific cost or price range, e.g. $25-$50, $50-$75, $75-$100, etc.

"A lot of info is necessary to enter—have a few up front questions to eliminate/streamline users—if they already have an idea of what they want. Otherwise they can go through the entire thing."

Similarly, when specifically probed regarding the potential absence of certain types of calling plans (e.g. network-to-network, family plans, shared plans, favourite-5 plans), new users generally agreed that this represents somewhat of a weakness in the tool. Those interested in these types of plans were prepared to enter more information in the profile builder to allow the tool to incorporate this information in the results. Similar to cellular users, new users recommended that they be able to opt in or out of these types of plans at the beginning of the profile builder so that those not interested in these types of plans not be forced to answer unnecessary profile questions.

In terms of allocating their incoming and outgoing long distance minutes across points of origin and destinations:

• Many understood the difference between the categories although some found their "local calling area" confusing.
• There was interest in inputting the incoming and outgoing information on the same screen.

Similar to cell phone users, many cell phone intenders did not notice that the allocation of long distance minutes had to equal the total number of long distance minutes they had specified on the previous pages. Participants simply recommended auto deduction as a possible solution, i.e. an automatic countdown of the minutes as the user completes the fields.

Output

Over half of the participants noticed the three tabs on the output page. Some viewed their profile tab and only a few noticed the ability to increase/decrease their profile by 20%. Once brought to everyone's attention, all agreed this was a good feature. Similarly, they liked being able to change their profile on one page at the end. These features do need to be more prominently displayed though as less than half of the cell intenders noticed this.

Revised New User Flow (hybrid version)

After conducting the initial Cell Intender groups in Vancouver, it was decided that participants would be shown an alternate mock-up flow of the calculator instead of completing it as both a user and a non-user. This hybrid version took into account many of the comments and reactions from the Vancouver sessions, including:

• Providing user segments to help new users categorize themselves upfront—this segmentation would then drive typical usage patterns for some of the input requirements on subsequent pages;
• Merging ingoing and outgoing data requirements on one page; and ultimately,
• Reducing the number of screens new users would need to complete before reaching their profile summary and the plan details.

Participants were each given a 4-page handout of the proposed new flow. The website itself was not yet functional, so screen-shots were used to present the revised calculator to participants. The moderator then walked participants through each page via overhead projection at the front of the room and discussed various elements of the calculator. Importantly, participants were encouraged to make comparisons between the new flow and the one they experienced on the computer.

Overall perceptions

Overall, participants strongly preferred the new flow to the original one. They particularly liked the user-profiles on the first page, the layout, the lighter data entry requirements and the clearer step-by-step flow of the tool.

Page 1: Basic Information

All participants appreciated having the user profiles on the first page of the calculator (e.g. emergency only, occasionally, regularly or as a primary line of communication). Many were unfamiliar with how many minutes they would use on a cell phone, but almost all were able to self-identify with one of the new user profiles. Some found it to be a valuable learning tool in that it taught them about how many minutes are appropriate for various user profiles. A few participants noted that an additional category between "emergency use" (50 mins/month) and "occasional use" (350 mins/month)—or the option to pick both—and a category to capture usage exceeding 1,000 minutes would be helpful. In the absence of these additional categories, participants suggested an "other" category allowing them to enter a specific minute usage.

The majority of participants preferred the "Did You Know" box located at the bottom of the page, stating that it improves the overall flow and design of the page. Those that felt they did not really need this information preferred the box on the bottom, whereas those who relied on the information to complete the questions felt the box should be made more prominent and more associated to the questions (i.e. next to the question it pertains to). A suggestion for these instances might be to incorporate the information in with the instructions.

« Je suis moins obligé à le lire; il est là s'il me faut. Mais pour ceux qui ont vraiment besoin de cet info, c'est plus difficile de le trouver en bas de page »

Page 2: Local Incoming/Outgoing Minutes

Compared to the computer version of the calculator, participants appreciated having incoming and outgoing minutes grouped together on the same page. Since many of them did not realize the difference the first time they used the calculator, they found this layout was much more clear and easier to use.

The first step on this page was seen as much more clear and straightforward than the original version. Participants liked how default numbers were already in the boxes, and almost all realized that this was a function of the user profile they had previously chosen. Participants liked the idea of playing around with the calculator and testing out different usage scenarios. While most realized that the number allocation could be changed, a few thought the pre-filled "total" could not be changed. Some felt the ability to change all numbers, including the total, should be more explicit in the task instructions. One participant suggested that the weekday/evening/overnight/weekend distribution automatically adjust when the total is changed.

"It allows you to easily play around with different scenarios"

"Il est simple, clair, et pas compliqué"

However, as was seen in the computer version, the breakdown of evening minutes by hour in the second task on this page was too precise, and a difficult task for most participants. Again, this was attributable to a general misunderstanding of how service providers structure their offers.

"It's ridiculous to break down minutes like this; at best, it would be a guess"

Comments from participants would suggest that they were overly concerned with the precision of the data they entered—they did not seem to realize that an estimate was all that was needed. Perhaps there is a preconception or preoccupation that the accuracy of their data will drive the accuracy of the tool—a concern along the lines of "garbage in, garbage out."

Some suggested offering a "typical usage profile", as was done in the other version of the calculator. Others thought that entering percentages instead of absolute numbers would be simpler. Interestingly, participants felt they would be more willing to put in the time to distinguish hourly usage if they understood the reasoning behind it. After reading the "Did you know" box many felt the task was necessary and they realized they could benefit from entering the precise data. Their patience and willingness to devote time to the task increased immensely. Given this, it is suggested that this information be placed closer to the task or that a brief explanation be incorporated into the statement by the calculator icon.

« Ça change tout si on sait pourquoi il faut le décortiqué comme ça »

Regarding the task asking for the percentage of the total monthly minutes used for outgoing calls, nearly all participants felt this step was clear, easy to think about, and they appreciated having the typical usage as a default number.

Page 3: Long Distance Incoming/Outgoing Minutes

The majority of participants were quite comfortable estimating their incoming and outgoing long distance minutes. They liked the layout of the table, especially having boxes for incoming and outgoing calls right beside each other. However, many were unclear if they were supposed to enter the number of minutes or the number of calls. They noticed an inconsistency between the section title, the table footer and the task text (which all ask for minutes) and the column headers (which indicate calls). Participants should also be reminded that the totals on this page (and where ever else this applies in the tool) are monthly figures, e.g. "Total long distance minutes per month".

An important challenge for some participants is the definition of their "local calling area"—they would not know what their footprint is. This was especially a problem in Montreal where calling off-island may or may not be long distance, depending on your local area and where you are situated when making a call.

Participants were fairly comfortable breaking down their long distance minutes by weekday/evenings/overnight/weekend, but some suggested using a percent allocation instead. While the numbers should add up, site users should be reminded from time to time that the entire calculator task is an exercise in approximation and that users should enter numbers to the best of their ability.

Not having international calling as an option was not a problem for the majority of participants. Most agreed that they would be very unlikely to use their cell phone for international calling, with the exception of one woman who said she would cancel her landline if/when she gets a cell phone and therefore need access to International calling on her cell phone. Others thought that the question could be asked up front, as is done with the general long distance question, and an international section could be added for those who required it.

Page 4: Features

For most participants, the last page of the hybrid version was simple and straightforward. Some participants did not realize that the question mark was a link to additional information about each service. As such, participants who were unfamiliar with some of the services had trouble completing this section. To make this help feature more evident, participants suggested:

• Converting the service name into a hyperlink;
• Adding a legend up front so that the user knows what the symbol means;
• Using a different symbol (more like the smiley face used for the "did you know") rather than a question mark—some sensed the question mark was the punctuation in the text rather than an actual link to more information; and,
• Having a mouse-over definition instead of a pop-up.

Participants were also concerned about how the calculator would interpret bundled services. They would be interested to know the incremental value of each service if it were part of a bundle. For example, if you checked off voice mail and caller ID, could the calculator tell you that for only an additional $1 or $2 you could also get text messaging because it is part of a bundled offer? Participants would also like to know the approximate monthly cost for these services so they would know whether or not to include them in their usage profile right away rather than wait until the results pages to understand the cost impact of these features. As such, some suggested simply showing a monthly price range beside each feature.

Some participants noted that there was no space for incoming text message—since the group was with non cell users the majority of them did not know that incoming text messages are free on most cellular plans. In fact, some current cell phone users did not know this!

Some also suggested adding in other features such as Internet/email/data transfer so that they could see how different options would affect the price of the plans (i.e. is it worth it to get the additional features?). Although most admitted they would not actually want Internet service on their phone, they were curious to see how it would change the overall price of their plan. As well, some noted that because the industry is always evolving, some of the features considered in the calculator (e.g. voice mail, SMS, etc.) will likely soon become standard features and that other options such as mobile TV, Internet browsing, etc. represent the future and should be considered.

Summary Assessment

Despite the various criticisms made of the tool among both cell phone intenders and cell phone owners, the vast majority of participants would actually use the tool if it were available. As well, many participants would likely mention it to friends and family if they were in the market for a cell phone plan. They would describe the tool as:

• providing options from all the providers in you area;
• providing detailed information on a variety of plans;
• helping you find out if you're spending too much on your current plan;
• it does all the work for you;
• the results are personalized based on actual usage;
• allowing you to compare different plans side by side.

"I think with your own phone bill and knowing generally on how you use you phone during the month it's pretty easy to use."

« Il nous plonge dans le contexte réel des cellulaires aves les variables nécessaires »

A very select few, most of whom sensing they would not make much use of the proposed tool, questioned the Government of Canada's involvement in this type of initiative. They were not convinced it was their role to help Canadians shop for a cellular phone plan. These points were typically counter-argued by others in the sessions, who in fact understood and appreciated the consumer education and protection role this tool actually served.

In terms of marketing, some of the more common ways participants thought the government could inform consumers of this website included:

• Commercials on TV, like the Service Canada commercials (which were frequently recalled by participants);
• Place signage at points of purchase where ever you can purchase a cell phone plan;
• Google search results (based on keyword searches on "compare cell phone," "cell phone plans", etc.);
• Include as an insert with the cell phone bill;
• Include as a link on cell phone company websites;
• Ads at bus stops/ shelters and in newspapers; and,
• Government of Canada website pop-up advertisement.

Summary Recommendations

Key Issues	Recommendations
General Comments
Concerns with business usage	Make language on the home page more explicit regarding how the user is to consider this tool
Location of the "Did you know" boxes—distracting, forced users to scroll, not always relevant depending on user knowledge	Move all boxes to the bottom of the page. Where the information was deemed useful by users to complete a task, it might be preferred to integrate the information in the task instruction.
Interest in seeing plans without having to go through the profile builder	Make the link to this information more evident on the home page
Concerns that the information is not updated	Make the "Last updated" information more evident on the plan details pages and on the home page
Basic Information Page
Don't know the difference between prepaid and monthly plans	Provide an explanation on the appropriate page in the profile builder or add a link to more information
No information on international calls	Add to calculator but allow funneling at the beginning of the profile builder
No information on specific types of plans (e.g. family plans, shared plans, etc.)	Add to calculator but allow funneling at the beginning of the profile builder
Want to pre-filter their profile for specific price ranges	Add a filter at the beginning of the profile builder (e.g. $25-$50, $50-$75, etc.)
Local Minutes Section
Concerns with level of detail asked around minute allocation	add more context to the task description allowing the user to appreciate why the detail is needed merge pages together wherever possible, especially ingoing and outgoing minutes
Confusion with ingoing versus outgoing data	make labels identifying ingoing and outgoing more explicit / larger/ more evident merge pages together wherever possible
Lack of understanding of what the "Calculate" function does	Remove this function altogether, or, Enter typical usage patterns by default
"Calculate" function returns user to top of the same page	Remove this function altogether, or, Return user to the same location on their page rather than resending them to the top of the page
Moderate to low confidence in the typical usage patterns	Allow users to select from a list of pre-defined user segments (similar to the new hybrid flow)
Long Distance Minutes Section
Concerns with level of detail asked around minute allocation	add more context to the task description allowing the user to appreciate why the detail is needed merge pages together wherever possible, especially ingoing and outgoing minutes
Confusion with ingoing versus outgoing data	make labels identifying ingoing and outgoing more explicit / larger/ more evident merge pages together wherever possible
Pre-populated zero fields—errors because left blank, and participants not deleting the zero before entering their value	have all fields blank by default and accept a blank as a zero, or, have all fields blank by default and force a value for all fields, or, highlight the entire field when clicked on allowing the user to easily overwrite the default value
Minute allocation not equaling the prefilled totals, especially for long distance minutes	use typical usage patterns by default use auto-deduction make the user's total more evident in the task instruction (e.g. Allocate your 50 long distance minutes…)
Lack of understanding of what the "Calculate" function does	Remove this function altogether, or, Enter typical usage patterns by default
"Calculate" function returns user to top of the same page	Remove this function altogether, or, Return user to the same location on their page rather than resending them to the top of the page
Moderate to low confidence in the typical usage patterns	Allow users to select from a list of pre-defined user segments (similar to the new hybrid flow)
Confusion regarding "local calling area"	Provide users guidance on how they can find out about how their local calling area is defined and / or, Instruct users what assumptions they should make about their local calling area to help them enter estimates in the profile builder
Difficult to allocate long distance minutes	Consider using a percent allocation instead of asking for specific minute allocation
Features Section
Features list—« suivi d'appel » not well understood in French	Change to « Transfer d'appel »
Features list—links to more information not recognized as links	change the icon provide information via mouse-over provide information directly on the page beside the feature provide information in a "Did you know" box
Interest in feature prices before getting to plan details	Provide approximate monthly rates beside each feature (e.g. Voice mail—app. $5/month)
Profile Display
Location of "Submit my Profile" button	Switch the location of this button with the "Print" button
"Change my profile" not always noticed	Make the editing of the user profile more prominent—larger buttons / links
Results Display
Sort functions in plan details not always noticed	Make visuals more evident—larger icon
"Limit to single provider"—cannot easily return to All providers	Add an "All providers" option rather than having to select a blank to revert back to the list of all providers
Detailed Plan View
Comparing plans side-by-side lacks visual appeal	Use of colors or column structure to better distinguish the three plans being compared
Can only compare 3 plans side-by-side	Allow the user options to compare as many side-by-side, up to 5
Plan details missing	information on contracts provider coverage
Hybrid Version
Page 1—Time of day phone usage (task 4)	Move "All the above" to the top of the list and rename to "All time periods" (or something similar)—this way the user does not click all the options only to get to the bottom to find the all-encompassing choice
Page 2—Time of day minute allocation (task 1): not sure if numbers can be changed	Make instructions more explicit that the user can change the allocation and the total
Page 3—LD minute allocation (task 1): confusion over minutes and calls	Change column headers in the grid to as for "minutes"—this would make them more consistent with the page title, the instruction and the "total" label at the bottom of the grid

Appendix A: Moderation Guide

Agenda	Allotted Time
1. Introduction and warm-up	10 minutes
2. General context setting	10 minutes
3. Obtaining / Comparing Cell phone Plan Information	15 minutes
4. Hands-on Usability Testing	60 minutes
5. Value and Gap Assessment	20 minutes
6. Conclusions	5 minutes
Total	120 minutes

Cell Phone User Handout Material and Hyperlink Inventory:

• Handouts: 3 mocked-up cell phone bills
• Links:
  • Link to 2 retailer websites
  • Link to 3 cell phone provider websites
  • Link to a comparison website
  • Link to online calculator (pages to be specified by OCA)
  • Link to online calculator home page (for hands-on testing)
• CCIG / CC brochures to be handed out with incentive payout (provided by client team)

1. Introduction and warm-up (10)

Welcome. You are about to be part of an important process in assisting Industry Canada and the Office of Consumer Affairs. Your experience and insight are extremely important.

Before we begin, allow me to introduce myself. My name is Rick and I have been hired to lead this discussion with you. I am not an employee of the Government of Canada.

This evening I'll be asking you a number of different questions about a new interactive information product. It is important to remember that there are no right or wrong answers and that nothing you say will hurt or offend me. It is your honest opinion that counts.

During our discussion, I may take notes and I will audio record this interview. This is standard procedure for interviews of this nature. These audio tapes will only be used to help me recall details and report people's opinions accurately. Nothing you say or do will be identified to you as an individual and you will never be contacted in connection with this session. Anything you say is protected under the Government of Canada's Privacy Act and the Access to Information Act.

Members of the client team are at the back of the room to listen attentively to your comments. They will not be participating in the session, they are not here to answer your questions about the products we will be testing—they are simply here to observe.

We want this discussion to be open and frank. We want to hear your honest views and suggestions. If you have an idea, do not hold back.

Do you have any questions before we begin?

What I would like to do now is get to know you a little. Could you please tell me your first name, a little something about yourself, such as:

• What kind of work you do?
• If you're a student, what are you studying? Do you have a part-time job?
• Do you have any children, if so what age?
• Do you find yourself traveling a lot? … for pleasure or for work?

2. General context setting (10)

We're going to kick the discussion off with a general question: I'd like you to describe for me what it's like being a consumer these days.

2.1 In your opinion, what are some of the best aspects about being a consumer these days?

2.2 In your opinion, what's the hardest part about being a consumer these days? What do you have to work hardest at in order to get done as a consumer? What have you done to help put your mind at ease?

2.3 And what would you see as some of the challenges and advantages for consumers when it comes to the cell phone market specifically?

• … are there any issues that come to mind when it comes to shopping for a cell phone or deciding which cell phone plan you need? Probe as needed: What about the quantity of information available to consumers when they are shopping for a cell phone?
• … what about the ease of access to information?
• … what about the credibility of the information?

3. Obtaining/Comparing Cell Phone Plan Information (15)

As some of you may know, there are many ways for consumers to obtain information on cell phone plans. I'm going to present some different sources and I'll be interested in hearing from you what you see as the pros and cons of each one.

3.1 Cellular phone retailers: One way consumers can obtain information on cell phone plans is by visiting specific retailers that sell cell phones such as Future Shop and Best Buy.

• http://www.futureshop.ca (Future Shop)
• http://www.bestbuy.ca (Best buy)

3.2 Cellular phone company websites: You can also visit cellular phone service provider stores or websites to get information on plans. Moderator shows participants three bookmarked cellco websites—Rogers, Bell Mobility and Telus:

• http://www.rogers.com/web/content/wireless-plans (Rogers main wireless page)
• https://www.bell.ca/ (Bell main wireless page)
• http://www.telusmobility.com/ (Telus main wireless page)

3.3 Comparison websites: There are also websites specifically designed to allow consumers to compare plans from various cellular companies side-by-side. The sites typically have the plans presented in a grid format, with the plans across the top and the plan features along the left hand-side, allowing consumers to compare plans along the same features. One such site is Compare Cellular. Moderator shows participants 2 screens of the Compare Cellular website:

• http://www.comparecellular.com/bell-plans-and-features/ (Plan Detail)
• http://www.comparecellular.com/plans-and-add-ons/ (Side by Side)

3.4 Calculator website: Consumers could also use a website that enables them to enter some details about their cell phone usage. The calculator then compares this usage profile against plans available in the market across all the different cellular phone companies and estimates the monthly cost for plans available. Moderator shows participants 5 screens of the calculator website:

• www.ic.gc.ca/eic/site/oca-bc.nsf/eng/ca02270.html (Welcome Page)
• Profile Page
• Result Page
• Compare 3 Page
• Plan Details

After presenting each approach, the moderator will probe for the following:

3.5 Through a show of hands, who here has ever used this approach, or something similar, to obtain information on cellular phone plans?

3.6 What do you see as the main advantages of trying to obtaining information on cell phone plans this way?

3.7 What do you see as the main disadvantages, limitations or challenges when trying to obtain information on cell phone plans this way?

Specifically regarding the online calculator:

3.8 What are your expectations regarding an online calculator?
Probe as needed: Expectations regarding …

• … the amount of effort you would have to put into it?
• … the quality of the information it would provide you?
• … the quantity of information it would provide you?
• … how easy it would be to use the tool?

3.9 How much time would you be prepared to invest to go through this kind of calculator in order to view the plans that best suit your usage profile?

Summary perspective on sources of information:

3.10 Are there any other sources of information on cell phone plans that you or someone you know has used?

3.11 Out of the four means of obtaining information on cell phone plans we have discussed to date …

• … which do you find most practical or helpful?
• … which would you be tempted to use?
• … which would you be most likely to trust or recommend?

4. Hands-on Usability Testing (60)

The remainder of the session will now focus on a prototype or test version of an online calculator. Test versions are an important stage in the development of a website—by having potential users such as yourselves go through the online tool, it helps us understand how well the tool works. We won't be entering real information and the tool will produce simulated output based on fictitious service providers and cell plans. To help us test the tool, I will have each of you go through the calculator assuming you are interested in seeing what plans in the marketplace best suit your usage profile. To make sure we are all on the same page, we have prepared three mocked-up cellular phone bills that you will use to work your way through the calculator [Participants have 3 mock bills on their desk].

What I would like you to do is start with the bill from [randomize from city to city and from group to group] Galaxy / Symphony / Gem. [Moderator to show sequence on a flipchart or whiteboard to ensure all participants follow the same sequence.] Simply assume that this is your cellular phone bill.

I'd like you to use the information available on each bill to work with the calculator—you'll get about 10 minutes per bill. Once everyone is done the first bill, we'll have a quick discussion about your experience so please don't move onto the next bill until I say so.

As I said earlier, an important objective for me is to understand whether you can effectively use the tool. So I would encourage you to jot down some quick notes as you work your way through the calculator. Things I'll be interested in hearing from you include [Moderator to provide list on flipchart / whiteboard]:

• Likes and dislikes
• Any problems you encountered
• Any recommendations you would make based on what you've seen
• Questions you are asking yourself as you go through each exercise
• Clarity of the instructions

Try to make your way through the calculator as best you can. You shouldn't feel bad if you don't reach the end—this isn't a test and this isn't a race. If you feel you've hit a roadblock, try to keep moving forward even if the information you enter is not exactly what you think it should be—remember, you only have 10 minutes. Just take note of these roadblocks because it is important for me to understand the challenges you encountered.

After each scenario, probe on the following:

4.1 By a show of hands, how many made it to the results screens (i.e. where the calculator presents a list of plans that reflect the usage profile you've entered)?

4.2 What are your overall reactions to the calculator?

4.3 What, if any, are some of things you feel the tool does really well?

4.4 What, if anything, do you see as some of the tool's weaknesses?

4.5 What, if anything, would you do differently if you could redesign certain parts of the calculator?

5. Value and Gap Assessment (20)

Now that you've had a chance to go through the calculator a few times, lets get some summary perspective on your experience. Moderator to probe as needed, depending on what was covered in each post-scenario discussion in section 4.

5.1 Overall assessment:

• What, if any, are the main problems or difficulties you encountered along the way?
• What did you like about the tool?
• Are there any specific features on the website that helped you complete your task?

Quick questions!

5.2 Reactions to look and feel:

• What are your thoughts on the look and feel of the tool?
  Probe as needed: What are your thoughts on …
  • … the site's use of colors?
  • … the general layout of the pages?
  • How would you describe the balance of text and space on the pages? … are the pages too cluttered or just right?
  • What do you think of the "Did you know" boxes? Did you even read them? Are they useful / relevant? Do they add value? What about their location on the pages?

5.3 Reactions to navigation:

• What are your thoughts on your ability to navigate the site? Probe as needed: What are your thoughts on …
  • … how easy it was to move from one page to the next?
  • … the ease of understanding the purpose of all the links and buttons on each page?

5.4 Perceptions regarding level of effort:

• Do you feel the amount of information the calculator asks of you is fair? … do you feel it is necessary?
• Do you feel the amount of time it took you to complete the task is fair considering the results the calculator generates?
• Did any of the steps seem redundant or unnecessary to you? If so,which ones?

5.5 Reactions to output:

• What do you think of the results you get? Probe as needed:
  • What are your thoughts on the amount of information provided?
  • Do you feel you are being provided the kind of information that allows you to make an informed decision about cell phone plans?
  • What do you think of the way the plans are presented?
  • What do you think of the ability to sort the results and the ability to compare plans?

5.6 Final assessment:

• Based on what you've seen, what would you consider to be the key strengths of this online tool?
• Based on what you've seen, what would you consider to be the key weaknesses of this online tool? If you had to prioritize one specific improvement, what would it be?
• Do you think that you would be able to go home and create your own usage profile using your own cell phone bill?
• Is there anything you would expect or like the site to do but you feel is not being addressed? Are there any aspects of cell phone plans you feel have not been considered by this tool? Which ones? Probe as needed: What if I told you the calculator does not consider …
  • … international calling other than the US?
  • … certain types of plans such as Family Plans, Shared plans and Network to Network plans?
  • … the cost of acquiring the handset?
  • … certain features and options?

For each missing feature, probe:
Would you consider this to be a significant weakness in the tool or do you feel you would still get value from the tool without this dimension being considered? If feels it is a significant weakness: What if I told you that you would need to enter more usage information in order for the tool to consider some of these dimensions you feel are important. Would you be willing to answer additional questions in order for the tool to address these issues? Do you think the advantages would outweigh the level of effort needed to obtain the information?

What if the final tool doesn't address these issues? What do you think the consequences might be?

6. Conclusions (5)

6.1 How would you describe this online tool to friends and family tomorrow?

6.2 How would you want to learn about this tool? How do you think it should be promoted? What do you think is the best way for the Government of Canada to bring a website like this to your attention?

6.3 Once launched, do you think you will access this website again?

• Why/why not?

6.4 Do you have any additional comments about the site?

6.5 Are there any questions or issues that should be passed on to the government department that created this website?

Thank You! Those are all the questions I have for you tonight. The young lady/ man who asked you to sign-in will ask you to sign-out. This is just to confirm you received the incentive we promised you! Good night!

Agenda	Allotted Time
1. Introduction and warm-up	10 minutes
2. General context setting	10 minutes
3. Obtaining / Comparing Cell phone Plan Information	15 minutes
4. Hands-on Usability Testing	60 minutes
5. Value and Gap Assessment	20 minutes
6. Conclusions	5 minutes
Total	120 minutes

Cell Phone Intender Handout Material and Hyperlink Inventory:

• Handouts: 3 lifestyle scenarios
• Links:
  • Link to 2 retailer websites
  • Link to 3 cell phone provider websites
  • Link to a comparison website
  • Link to online calculator (pages to be specified by OCA)
  • Link to online calculator home page (for hands-on testing)
• CCIG / CC brochures to be handed out with incentive payout (provided by client team)

1. Introduction and warm-up (10)

Welcome. You are about to be part of an important process in assisting Industry Canada and the Office of Consumer Affairs. Your experience and insight are extremely important.

Before we begin, allow me to introduce myself. My name is Rick and I have been hired to lead this discussion with you. I am not an employee of the Government of Canada.

This evening I'll be asking you a number of different questions about a new interactive information product. It is important to remember that there are no right or wrong answers and that nothing you say will hurt or offend me. It is your honest opinion that counts.

During our discussion, I may take notes and I will audio record this interview. This is standard procedure for interviews of this nature. These audio tapes will only be used to help me recall details and report people's opinions accurately. Nothing you say or do will be identified to you as an individual and you will never be contacted in connection with this session. Anything you say is protected under the Government of Canada's Privacy Act and the Access to Information Act.

Members of the client team are at the back of the room to listen attentively to your comments. They will not be participating in the session, they are not here to answer your questions about the products we will be testing—they are simply here to observe.

We want this discussion to be open and frank. We want to hear your honest views and suggestions. If you have an idea, do not hold back.

Do you have any questions before we begin?

What I would like to do now is get to know you a little. Could you please tell me your first name, a little something about yourself, such as:

• What kind of work you do?
• If you're a student, what are you studying? Do you have a part-time job?
• Do you have any children, if so what age?
• Do you find yourself traveling a lot? … for pleasure or for work?

2. General context setting (10)

We're going to kick the discussion off with a general question: I'd like you to describe for me what it's like being a consumer these days.

2.1 In your opinion, what are some of the best aspects about being a consumer these days?

2.2 In your opinion, what's the hardest part about being a consumer these days? What do you have to work hardest at in order to get done as a consumer? What have you done to help put your mind at ease?

2.3 When we asked you a few brief questions over the phone to invite you to this evening's sessions, you indicated that you were interested in getting a cell phone in the near future. What would you see as some of the challenges and advantages for consumers when it comes to the cell phone market specifically?

• … are there any issues that come to mind when it comes to shopping for a cell phone or deciding which cell phone plan you need? Probe as needed: What about the quantity of information available to consumers when they are shopping for a cell phone?
• … what about the ease of access to information?
• … what about the credibility of the information?

3. Obtaining/Comparing Cell Phone Plan Information (15)

As some of you may know, there are many ways for consumers to obtain information on cell phone plans. By a quick show of hands, who here has already done some shopping around for a cell phone service? Has anyone here previously owned a cell phone?

I'm going to present some different sources and I'll be interested in hearing from you what you see as the pros and cons of each one.

3.1 Cellular phone retailers: One way consumers can obtain information on cell phone plans is by visiting specific retailers that sell cell phones such as Future Shop and Best Buy.

• http://www.futureshop.ca (Future Shop)
• http://www.bestbuy.ca (Best buy)

If there is limited shopping experience in the room, the moderator will briefly explore each source of information via overhead projection—this will remain very high level—this is not mean to be a tutorial on cell plan information gathering!

3.2 Cellular phone company websites: You can also visit cellular phone service provider stores or websites to get information on plans. Moderator shows participants three bookmarked cellco websites—Rogers, Bell Mobility and Telus:

• http://www.rogers.com/web/content/wireless-plans (Rogers main wireless page)
• https://www.bell.ca/ (Bell main wireless page)
• http://www.telusmobility.com/ (Telus main wireless page)

3.3 Comparison websites: There are also websites specifically designed to allow consumers to compare plans from various cellular companies side-by-side. The sites typically have the plans presented in a grid format, with the plans across the top and the plan features along the left hand-side, allowing consumers to compare plans along the same features. One such site is Compare Cellular. Moderator shows participants 2 screens of the Compare Cellular website:

• http://www.comparecellular.com/bell-plans-and-features/ (Plan Detail)
• http://www.comparecellular.com/plans-and-add-ons/ (Side by Side)

3.4 Calculator website: Typically, individuals who are looking to sign up to a cellular service for the first time need to have some understanding of how they plan to use their cell phone—in other words estimate their anticipated usage profile. This would include having a sense of the number of local and long distance calls they would make from their cell phone, how many they would receive, their use of text messaging, etc. Consumers could use an online tool such as an interactive calculator—the user basically enters some details about their anticipated cell phone usage and the calculator then compares this usage profile against plans available in the market across all the different cellular phone companies and estimates the monthly cost for plans available. Moderator shows participants 5 screens of the calculator website:

• www.ic.gc.ca/eic/site/oca-bc.nsf/eng/ca02270.html (Welcome Page)
• Profile Page
• Result Page
• Compare 3 Page
• Plan Details

After presenting each approach, the moderator will probe for the following:

3.5 Through a show of hands, who here has ever used this approach, or something similar, to obtain information on cellular phone plans?

Although some of you may not have been familiar with this type of source of information and you are seeing it for the first time tonight, based on what you've seen so far …

3.6 … what do you see as the main advantages of trying to obtaining information on cell phone plans this way?

3.7 … and what do you see as the main disadvantages, limitations or challenges when trying to obtain information on cell phone plans this way?

Specifically regarding the online calculator:

3.8 What are your expectations regarding an online calculator?
Probe as needed: Expectations regarding …

• … the amount of effort you would have to put into it?
• … the quality of the information it would provide you?
• … the quantity of information it would provide you?
• … how easy it would be to use the tool?

3.9 How much time would you be prepared to invest to go through this kind of calculator in order to view the plans that best suit your usage profile?

Summary perspective on sources of information:

3.10 Are there any other sources of information on cell phone plans that you or someone you know has used?

3.11 Out of the four means of obtaining information on cell phone plans we have discussed to date …

• … which do you find most practical or helpful?
• … which would you be tempted to use?
• … which would you be most likely to trust or recommend?

4. Hands-on Usability Testing (60)

The remainder of the session will now focus on a prototype or test version of an online calculator. Test versions are an important stage in the development of a website—by having potential users such as yourselves go through the online tool, it helps us understand how well the tool works. We won't be entering real information and the tool will produce simulated output based on fictitious service providers and cell plans. To help us test the tool, I will have each of you go through the calculator assuming you are interested in seeing what plans in the marketplace best suit a usage profile. To make sure we are all on the same page, we have prepared three scenarios that you will use to work your way through the calculator [Participants have 3 lifestyle profiles on their desk].

What I would like you to do is start with [randomize from city to city and from group to group] Name 1 / Name 2 / Name 3. [Moderator to show sequence on a flipchart or whiteboard to ensure all participants follow the same sequence.]

I'd like you to read the first scenario and, based on the lifestyle information presented, work your way through the calculator by estimating as best you can how each person featured in the scenario might use their cell phone—you'll get about 10 minutes for each scenario. Once everyone is done the first scenario, we'll have a quick discussion about your experience so please don't move onto the next scenario until I say so.

As I said earlier, an important objective for me is to understand whether you can effectively use the tool. So I would encourage you to jot down some quick notes as you work your way through the calculator. Things I'll be interested in hearing from you include [Moderator to provide list on flipchart / whiteboard]:

• Likes and dislikes
• Any problems you encountered
• Any recommendations you would make based on what you've seen
• Questions you are asking yourself as you go through each exercise
• Clarity of the instructions

Try to make your way through the calculator as best you can. You shouldn't feel bad if you don't reach the end—this isn't a test and this isn't a race. If you feel you've hit a roadblock, try to keep moving forward even if the information you enter is not exactly what you think it should be—remember, you only have 10 minutes. Just take note of these roadblocks because it is important for me to understand the challenges you encountered.

After each scenario, probe on the following:

4.1 By a show of hands, how many made it to the results screens (i.e. where the calculator presents a list of plans that reflect the usage profile you've entered)?

4.2 What are your overall reactions to the calculator?

4.3 What, if any, are some of things you feel the tool does really well?

4.4 What, if anything, do you see as some of the tool's weaknesses?

4.5 What, if anything, would you do differently if you could redesign certain parts of the calculator?

Alternate approach consideration: Based on the "success rate" of the first scenario, the moderator may resort to a classroom / team-based approach to complete the next scenario if participants are open to this idea. The moderator will have participants guide each step in the process while the moderator works through the calculator via the overhead projector. Throughout, the moderator can ask participants to voice the following out loud as the group progresses from one screen to the next:

• Likes and dislikes
• Any problems encountered
• Any recommendations participants would make based on what they are seeing
• Clarity of the instructions

For the last scenario, the participants can decide whether or not to try it on their own or proceed "classroom style".

5. Value and Gap Assessment(20)

Now that you've had a chance to go through the calculator a few times, lets get some summary perspective on your experience. Moderator to probe as needed, depending on what was covered in each post-scenario discussion in section 4.

5.1 Overall assessment:

• What, if any, are the main problems or difficulties you encountered along the way?
• What did you like about the tool?
• Are there any specific features on the website that helped you complete your task?
• Were there instances when you had no idea of type of information you needed to enter? In other words, you did not understand what you needed to do …
  • Can you briefly describe those instances for me?
• Did you always know when you needed to enter required information?

Quick questions!

5.2 Reactions to look and feel:

• What are your thoughts on the look and feel of the tool? Probe as needed: What are your thoughts on …
  • … the site's use of colors?
  • … the general layout of the pages?
  • How would you describe the balance of text and space on the pages? … are the pages too cluttered or just right?
  • What do you think of the "Did you know" boxes? Did you even read them? Are they useful / relevant? Do they add value? What about their location on the pages?

5.3 Reactions to navigation:

• What are your thoughts on your ability to navigate the site? Probe as needed: What are your thoughts on …
  • … how easy it was to move from one page to the next?
  • … the ease of understanding the purpose of all the links and buttons on each page?

5.4 Perceptions regarding level of effort:

Probing specific functionalities:

• Did any of you notice the ability to access a "typical usage pattern"?
  • Was it clear to you what it meant?
  • Was it clear to you how it could be used?
• Was it clear to you when you needed to enter incoming call information versus outgoing call information? Was it clear to you how to enter that information?
  • How about when you needed to enter the distribution of minutes across specific time periods … did you understand what you needed to do here? How easy or difficult was this?
• Was it clear to you when you needed to enter long distance call information?
  • How about when you needed to enter the distribution of specific types of long distance calls … did you understand what you needed to do here? How easy or difficult was this?

Probing on overall level of effort:

• Do you feel the amount of information the calculator asks of you is fair? … do you feel it is necessary?
• Do you feel the amount of time it took you to complete the task is fair considering the results the calculator generates?
• Did any of the steps seem redundant or unnecessary to you? If so, which ones?

5.5 Reactions to output:

Moderator will show an example of an output page for all to see in case some did not make it to the results

• What do you think of the results you get? Probe as needed:
  • What are your thoughts on the amount of information provided?
  • Do you feel you are being provided the kind of information that allows you to make an informed decision about cell phone plans?
  • What do you think of the way the plans are presented?
  • What do you think of the ability to sort the results and the ability to compare plans?

5.6 Final assessment:

• Based on what you've seen, what would you consider to be the key strengths of this online tool?
• Based on what you've seen, what would you consider to be the key weaknesses of this online tool? If you had to prioritize one specific improvement, what would it be?
• Do you think that you would be able to go home and create your own usage profile based on how you would anticipate using a cell phone?
• Is there anything you would expect or like the site to do but you feel is not being addressed? Are there any aspects of cell phone plans you feel have not been considered by this tool? Which ones? Probe as needed: What if I told you the calculator does not consider …
  • … international calling other than the US?
  • … certain types of plans such as Family Plans, Shared plans and Network to Network plans?
  • … the cost of acquiring the handset?
  • … certain features and options?

For each missing feature, probe: Would you consider this to be a significant weakness in the tool or do you feel you would still get value from the tool without this dimension being considered? If feels it is as significant weakness: What if I told you that you would need to enter more usage information in order for the tool to consider some of these dimensions you feel are important. Would you be willing to answer additional questions in order for the tool to address these issues? Do you think the advantages would outweigh the level of effort needed to obtain the information?

What if the final tool doesn't address these issues? What do you think the consequences might be?

6. Conclusions (5)

6.1 How would you describe this online tool to friends and family tomorrow?

6.2 How would you want to learn about this tool? How do you think it should be promoted? What do you think is the best way for the Government of Canada to bring a website like this to your attention?

6.3 Once launched, do you think you will access this website again?

• Why/why not?

6.4 Do you have any additional comments about the site?

6.5 Are there any questions or issues that should be passed on to the government department that created this website?

Thank You! Those are all the questions I have for you tonight. The young lady/ man who asked you to sign-in will ask you to sign-out. This is just to confirm you received the incentive we promised you! Good night!

Appendix B: Participant Hand Outs

Lifestyle Scenarios

Robert Collin

Since Robert has retired, his life hasn't slowed down. Robert is now volunteering 4 days a week at the local library between 11 am and 2 pm. Before returning home, Robert always takes a couple of minutes to call his wife to see if she needs something.

Robert has two young grand-daughters that live in the same town. He takes a few minutes to call them two to three times a week to see how they are. Often it's easier to reach them when they just come back from school but Robert is often out of the house at that time.

Robert is also president of the Local Sport Association. The Association holds a meeting once a week. Since the meeting is only open to the board of directors, Robert finds himself receiving one or two calls meeting evenings from coaches, tournament organizers, and local sponsors wanting to raise issues with the association.

On the weekend, Robert loves riding his motorcycle with his friends. They usually stay around town but on occasion they like to go on a full day trip in other cities. His wife usually stays home to work on her paintings. Robert always calls a few times when he's out since he knows that she might worry when he's on the road.

Notes:

Joanne Philips

Joanne and her husband have been married for nearly 15 years now. They live in a modest one bedroom apartment in the suburbs. Last week Joanne accepted a new job as a security guard at the local university located downtown. The commute takes her approximately 30 minutes in each direction.

The job requires her to work four 9-hour rotational shifts a week from either 7 am to 4 pm or 3 pm to 12 am. While the job requires her to patrol campus, she also has her own office with a computer and phone line so that she can file her incidence reports and talk to University Admin Staff in other departments as needed.

Last fall Joanne's father developed a very serious heart condition. Over the holidays he suffered a heart attack and has been under special care in the hospital since then. While her father has been in hospital, Joanne has been in constant contact with the doctors at the hospital, either in person or over the phone. However, she is now worried about being inaccessible while patrolling campus.

Her husband suggested that she get a new cell phone for emergencies only. This would enable her to go about her day and still be reachable if serious complications occur with her father. Since the cell phone will only be used in emergencies she really is not looking to spend a lot of money on it.

Notes:

Marc Newman

Marc just graduated from university and has moved to the city to pursue a career in acting. He just rented a small bachelor apartment downtown. The apartment is a little smaller than he is used to but, convenient because he is rarely at home either as a result of his busy social life or his auditions.

His parents live in another province and he tries to call them at least once or twice a month for 30 minutes to an hour each time. He also has a younger brother who is studying abroad in Berlin who he tries to call occasionally.

In order to receive call-backs from his agent for roles he has landed, audition calls and to stay in touch with his family and wide array of friends, he is considering purchasing a cell phone and disconnecting his landline. He has other friends who use their cell phone as their main line and it has been a really convenient change for them. He is also eager to join in on all the text messaging his friends are exchanging on the latest gossip and directions to the best parties.

Notes:

---

Galaxy Mobility

Previous Balance
Payment—Thank You
Amount Due

Monthly Charges Period for Feb 02/08 to Mar 02/08

	Details	Amount
Mars 50	500 Weekdays, unlimited evening and weekends	$50.00
System Access Fee		$6.95
911		$0.75
Planet 10 Bundle	Call ID, 100 out texts and Voicemail	$10.00
Extended Hours 6-8		$7.00
Total Usage Charges		$36.65
Total		$110.95 + taxes

Summary of Usage

Description	Amount (min)	Cost ($)
Weekdays	356	$0.00
Evenings	104	$0.00
Weekends	244	$0.00
Canada LD	36	$18.90
Canada to US LD	15	$7.50
International LD	0	$0.00
Roaming	0	$0.00
Amount (events)
Text Message	155 out 204 in	$8.25
Text International	10 out -0 in	$2.00
Total		$36.65

---

Gem Mobility

Previous Balance
Payment—Thank You
Amount Due

Monthly Charges Period for Feb 02/08 to Mar 02/08

	Details	Amount
Emerald 30	300 Anytime	$30.00
System Access Fee		$6.95
911		$0.50
Total Usage Charges		$0.00
Total		$37.45 + taxes

Summary of Usage

Description	Amount (min)	Cost ($)
Weekdays	112	$0.00
Evenings 9pm-7am	54	$0.00
Weekends	74	$0.00
Canada LD	0	$0.00
Canada to US LD	0	$0.00
International LD	0	$0.00
Roaming	0	$0.00
Amount (events)
Text Message	0 out 0 in	$0.00
Text International	0 out -0 in	$0.00
Total		$0.00

---

Symphony Mobility

Previous Balance
Payment—Thank You
Amount Due

Monthly Charges Period for Feb 02/08 to Mar 02/08

	Details	Amount
The First Movement	1250 Anytime	$110.00
System Access Fee		$8.95
911		$0.75
Le Maestro	Voicemail and Caller ID	$8.00
Long Distance 100	100 Canadian Long Distance Minutes	$10.00
Total Usage Charges		$132.51
Total		$270.21 + taxes

Summary of Usage

Description	Amount (min)	Cost ($)
Weekdays	775	$0.00
Evenings 9pm-7am	252	$0.00
Weekends	104	$0.00
Canada LD	240	$59.85
Canada to US LD	102	$45.90
International LD	14	$26.46
Roaming	0	$0.00
Amount (events)
Text Message	2 out 0 in	$0.30
Text International	0 out -0 in	$0.00
Total		$132.51

Appendix C: Hybrid Version