This story appeared on Network World at:
http://www.networkworld.com/reviews/2005/050205-test-web-conferencing.html
Interwise leads field of 11 vendors in global test of voice/video over IP services
By Christine Perey and Network World Lab Alliance, Network World, 05/02/05
Web conferencing vendors are adding two key features to their baseline data collaboration platforms: voice and video over IP. The advantages of being able to establish a single session for voice, video and data are increased productivity, reduced costs and the ability to produce integrated meeting archives. We tested 11 Web conferencing services from test sites in Switzerland, the U.K., Austria and the U.S. over the course of three weeks to get an "In the Wild" look at the latest services.
There are dozens of Web conferencing services to choose from (see our Buyer's Guide for the latest list). We focused on hosted services that are accessible from anywhere on the Internet, and that offer users with a telephone and PC-equipped headset and Web camera a multi-party, integrated voice and/or video-over-IP conference with several presentation, screen/file viewing and data collaboration features. Out of 20 companies invited, we tested 11: Centra, Convoq, Elluminate, Face-to-Face Meeting, Genesys, Interwise, Linktivity, Macromedia, Raindance, VidiTel and WebEx Communications .
Interwise's hosted Enterprise Connection Platform (ECP), iMeeting Version 5.2, wins the Clear Choice Award for its rich feature set, yet easy-to-use Web conferencing service. It delivered a flawless performance every time we initiated a rich media meeting with people who had (and who hadn't) previously downloaded the software and offered optimal levels of management and security for enterprise IT managers. Interwise also was the only service tested in which participants on the same conference could choose to use either the built-in, high-quality VoIP system or a traditional public switched telephone network (PSTN) connection to an Interwise-managed bridge. Using Interwise's softswitch (software bridge), the PSTN and VoIP participant voice qualities are indistinguishable and seamlessly mixed for any size meeting. Regardless how you choose to transmit voice, ECP Connect offers responsive desktop-, application- and file-sharing (or showing) for people on networks with different bandwidths, well-integrated whiteboard tools, co-browsing and simple polling. While the video over IP needs some work, it was another flexible feature in ECP-based meetings.
We were very impressed by the quality of several other services. Linktivity, Raindance and WebEx were excellent and are expected to implement improvements in the months to come.
Why go with a hosted service? http://www.networkworld.com/reviews/2005/050205-test-web-conference-side.html
Businesses can purchase Web conferencing in three different options: They can deploy it as an in-house application on the corporate network; a dedicated server can be set up and configured for a corporation on a hosting service's server; or the company can purchase seats or accounts on a third-party hosted application server shared with numerous other account holders. For companies with a high Web conferencing usage, especially among employees of the same company, purchasing and operating a server internally is frequently the most cost-effective scenario. Controlling secure access to the server behind a firewall for users on the public Internet can be challenging to set up and manage.
The fully hosted "open" yet secure services tested for this story offload all management and maintenance for a fixed monthly cost and are optimal when participants are widely distributed, all connected by way of the Internet and don't share a common IT infrastructure. Small and midsize businesses find this purchasing option most attractive. We also believe that application service providers can add features incrementally to their customers more quickly than they release them for their customer premises equipment customers. As the Web conferencing service providers reach a common baseline platform, including VoIP and support for video, and they begin to pay off their infrastructure investments, they increasingly will compete for customer business on the basis of price, making this option more financially attractive for any size customer.
Who played, who didn't http://www.nwfusion.com/reviews/2005/050205-test-web-conference-side2.html
The Web conferencing industry is crowded. More than 40 companies offer services for business users. We expect that there will continue to be consolidation in the industry in coming months and years.
To focus on the features we believe are most valuable for collaborative teams, we stipulated strict criteria for inclusion (we required that the service must have at least one real-time media, voice or video over IP, integrated with the full Web conferencing application). We extended invitations to 20 vendors: Centra, Cisco, Citrix, Convoq, Elluminate, Face-to-Face Meeting, Genesys, IBM, Interwise, Linktivity, Macromedia, Microsoft, NetSpoke, Pixon, Raindance, Viack, Santa Cruz Networks, Wave 3 Software, WebEx Communications and WiredRed. Fourteen companies responded favorably to the initial invitation, and the 11 finalists (see main story ) met our criteria for the tests.
Cisco and IBM said while their service platforms met the original criteria, their business model is to host a dedicated server on behalf of a corporate customer. Microsoft Live Meeting declined because it said that its target customers are customers with very large events, although the company expects to release companion services for collaborative teams in the near future.
Wave 3 Software's service was unavailable because of technical difficulties, preventing test team member access during the testing period, and therefore was not studied. Citrix didn't complete the testing because the service doesn't have real-time media fully integrated into the platform as we had originally understood.
Web conferencing test: How we did it http://www.nwfusion.com/reviews/2005/050205-test-web-conference-how.html
The testing team consisted of 11 people in four countries - the U.S., U.K., Switzerland and Austria. Testers used Windows XP-based laptops connected to either corporate networks or broadband Internet; others used Macintosh PowerBooks although support for Macintosh or Linux wasn't a criteria for inclusion in the study. Testers were provided Logitech Notebook Pro Webcams and Logitech Internet Chat Headsets, graciously loaned by Logitech Europe. Each service provided test member accounts, and test team members installed the client application or plug-ins on their clients.
After introductory briefings with service providers, test team members conducted hands-on tests on all components of personal multimedia conferencing and collaboration services in sub-teams of four participants. The four-member teams simulated the preparation of project documents and presentations over the Internet and their corporate networks.
Each service was evaluated for support of interactive voice communications (not quality but connectivity, point-to-point and multipoint), video communications (bidirectional not required but recommended), document viewing, document sharing (modification by user and/or host) and text chat.
When not in sessions, the evaluation team compared features that assist community members to communicate availability (state, presence), create and send multimedia messages, or schedule and initiate live sessions on a reservation and ad hoc basis.
Testing was conducted from March 9 to March 30, 2005.
All services we tested offer a live and responsive "shared space," where small group collaboration can occur (as opposed to a one-to-many event), and all services support at least one real-time video window. Eight of the 11 tested currently offer VoIP as an integral part of the conference (Raindance, Genesys and WebEx did not offer VoIP as an option at the time of testing. Seven of the 11 also offer meeting recording (Face-to-Face Meeting, Convoq, Raindance and VidiTel do not currently archive meetings).
Getting up and running
To deliver the promise of higher employee productivity, rich media conferencing needs to save time when compared with normal ways of conducting business. Although only 10% of the final score focused on setup, foolproof installations and usage in several environments (differing networks, operating system, hardware and software settings) is important for user acceptance and the financial viability of a service provider.
Seven services tested required Java and/or ActiveX for host application installation, usage or both. We found that certain browser settings, such as anything that blocks ActiveX or cookies, can interfere with installation. By extension, we found that Centra only uses Microsoft's implementation of the Java Virtual Machine (not the Sun version). We believe reliance on Microsoft-specific extensions caused an issue with Elluminate and Centra when we tried to enter a meeting with non-Windows browsers. Administrator rights might be needed to make changes to client computers before installation goes smoothly. Another little secret we found during installs was that the services that use ActiveX and Java within browsers can heavily use temporary Internet files. If the application doesn't automatically manage the temporary file cache, users can hit a memory ceiling and encounter difficulties during installation or usage.
Users don't always have rich media input devices (headsets and Web cameras) plugged into their PCs when they click on a meeting link. The result can be a lack of video or audio when entering a meeting with a platform that loads the media only at the start of one. Although we found a drawback with how it deals with video, Interwise's iMeeting was very adept at detecting and adapting quickly on the user's behalf to changes in media inputs and network connectivity. (We discovered that if a user is disconnected during an iMeeting, the application automatically tries to renegotiate with the same or a different server on the peered network grid on behalf of the user.)
Setup went extremely smoothly with Linktivity, Raindance and VidiTel, and we also got great speed and ease of use from services that use Flash (Convoq ASAP and Macromedia Breeze). Our testing team members connected to the hosted servers with anything from 128K to 1.5M bit/sec (the majority of the testers were on 1.5M bit/sec or higher). Based on regional differences (Europe vs. the U.S.) and possible connectivity bottlenecks between continents, we felt the hosting service providers' ISPs (rather than the services themselves) differed significantly in how well they served our needs. Although several test team members completed it on the first attempt, the longest set-up time required 8.5 minutes with Face-to-Face Meeting. We also found it irritating that WebEx required a partial re-install of the application each time a user or host enters a meeting.
Special features
Offering optional features (such as PowerPoint conversion and scheduling capabilities) in the form of plug-ins that can be installed separately as user needs evolve was a good strategy, exemplified by Interwise, Linktivity, Convoq and Macromedia. We also feel that user account naming conventions should be intuitive to users such as using family names or e-mail addresses. We dislike the legacy (PSTN-centric audioconferencing) feeling of numeric codes Genesys and Raindance use for identifying user accounts. In both cases, the user is issued a seven- to 10-digit meeting code (the same number, the user ID and meeting code, is stamped on a plastic card mailed to the user) and a four-digit PIN.
Getting people into a meeting at the appropriate time is critical to the success of a rich media conference. Services that support meeting scheduling (all those tested integrated scheduling through a portal or in conjunction with Outlook, with the exception of VidiTel and Face-to-Face Meeting) must always keep track of participant time zones. When populating an invitation for a meeting to be scheduled in the future, the service should ask what time zone the invitee is in. In some services this can be set once and saved on the server. In others, the invitation list is generated based on e-mail addresses, and there are neither time zones registered nor conversions offered. We also discovered that daylight-saving changes one week earlier in Europe than in North America, which Interwise handled gracefully but confused Genesys (we didn't test all the services on that week to see if others had correctly programmed their time zones). Elluminate Live requires that users convert from their time to Mountain Time. Integration with Exchange and Outlook and/or Lotus Notes simplifies this task, and in a corporation will offer a way to check participant availabilities and resolve phone numbers and e-mail addresses.
Most of the Web-based (service portal) meeting scheduling was simple to use, but in a few cases the numerous options made for an excessively long process; for example, the WebEx full-feature scheduling wizard has at least nine steps. Genesys' scheduling was equally long. Convoq ASAP doesn't offer a way to take advantage of a user's local Notes or Outlook calendaring features, but it is the only service to go the extra mile in the other direction. ASAP tracks an attendee's presence (away, available, busy, offline) by monitoring and integrating his public instant-messaging service buddy lists and the enterprise IM/presence management systems in its main user interface, which resides in the Windows system tray when not in use.
We lived through the pain customers experience when they must rely solely on audioconference bridges for the voice portion of their meetings. For example, getting testers in Switzerland and the U.S. to hear one another on the same conference bridge without intercontinental toll charges was a significant challenge with Raindance and Genesys. Both could offer dial-out services (for which someone ultimately must foot the bill) but users have to correctly enter their number including country access codes, and toll-free U.S. numbers can't be dialed from international locations. Sometimes a "+" symbol, a space or a parenthesis can set you back to the process of entering a phone number. In one instance, after about 15 minutes we managed to begin our meeting but we were short a participant who only could attend for the first short period before needing to attend another meeting.
To reduce the likelihood of a delayed meeting because of participant difficulties with the meeting software, Raindance meeting hosts can request live telephone customer support (an operator available via the integrated audioconference bridge) in 15 seconds or less.
We found it easy to start an ad hoc meeting from Outlook or a slim dashboard application in Genesys, Raindance, WebEx, Linktivity, Macromedia, Convoq and Interwise. It also is very easy to start a conference if you and the other meeting participants have the Face-to-Face Meeting and VidiTel applications running. Perhaps reflecting their heritage in the large event and educational worlds, we had to go through a portal (engaging a multistep process) to start any type of meeting in Centra and Elluminate.
Services differ widely in their ability to invite a new (ad hoc) participant into a meeting once the session has begun. We like the options Centra, Convoq, Raindance, WebEx, Face-to-Face Meeting and VidiTel offer to add a participant from within the application using an integrated IM invitation and the optional e-mail invitation. Genesys has a concept of an "Xpress meeting" from which ad hoc participants easily can be added by way of IM (but this feature is not available in the "normal" Genesys meeting). To invite someone when you are already in a Macromedia Breeze session, you must generate a new e-mail (Breeze provides a pre-configured template) or manually paste the URL into an IM. Because our testers weren't required to all be on the same public IM service, this could get to be a lot of cutting and pasting. Unfortunately, you can't use the application or service to invite someone in an ad hoc automated fashion when you are in an Elluminate meeting (you still have the option of pasting the URL into an IM or e-mail manually during the meeting).
Behind closed doors
As a matter of corporate policy or individual project needs, companies often have reservations about rich media conferencing over the Internet, or using hosted services, on the grounds of security and privacy.
Service providers ensured a high level of security in several ways. One way was to dedicate a server per enterprise customer, firewall each server and establish a VPN connection with the corporation. This method reproduces a premises-based system, and the service can manage only the server and features. Because our 16-member testing team was distributed across 14 Internet domains in the U.S. and Europe, we chose not to test this network configuration, and only tested services that are accessible on the public Internet.
We scored the services based on a comparison of the precautions offered by the services such as randomly generated meeting codes and passwords, the ability to lock meetings and require registration for entry. We didn't try to hack into meetings uninvited, break user password codes, eavesdrop on the encrypted media or take other steps to test the security in our real-world meetings.
All the services tested use proprietary signaling mechanisms to permit media to cross firewalls and to resolve calls between network address translation users (all test team members used private IP addresses). All the services tested also use SSL to encrypt session signaling and content.
Servers for Centra, Interwise, Linktivity, Macromedia and WebEx generated a unique meeting ID for each meeting. Others, including Convoq, Elluminate, Linktivity, WebEx and Genesys, give the meeting host the option to require those invited to reply by e-mail to register for the session. We found that virtually all the services can set a meeting password for participant use upon entry, with the exception of Face-to-Face Meeting and VidiTel, which use a call-me method for initiating meetings.
Genesys and Raindance have a waiting room where all meeting participants wait for the host to approve them before they attend the live meeting. Otherwise, we didn't find the level of security built into these two services very well adapted to the Internet, reflecting the heritage in traditional telephony. Eight of the 11 services (including Genesys and Raindance) also let the host press one button to lock a meeting room, preventing new participants from entering until a meeting is unlocked.
Beyond SSL, Face-to-Face Meeting, Interwise, Linktivity, Raindance and VidiTel add 128-bit encryption support. Others also might have additional encryption options but the ordinary user (whom the testers in this study represent) couldn't detect or make modifications to security settings from the portals or applications.
As mentioned earlier, seven of the 11 services tested permit hosts to record the meeting. Because of this, it's possible that meeting contents can be leaked (or shared) after a meeting. We feel the ability to archive a meeting holds a great deal of promise for information worker productivity. And provided the assets are managed securely on a service's network, we don't believe this presents a significant threat or corporate security or privacy. In fact, meeting-recording features might help companies comply with federal or industry-specific regulations regarding fair disclosure and provide the basis for e-learning.
Different ways to get work done
Once deployment, security and learning curves are addressed, employees should use technology to focus on business needs. Web conferencing services we tested fell into one of two camps (with the exception of Interwise and Linktivity, which seem to have negotiated a compromise). First, services such as WebEx and Centra, which grew in response to the needs of one presenter communicating to many in which people produce large virtual events (also known as Webinars) and expose participants to the same information at the same time. At the very minimum, this technology usage scenario requires robust invitation and entry management systems, support for polls and surveys, and a scalable network for a large number of participants to see the same information. The audience members need an extremely simple interface while the meeting host interface must provide crowd management functions such as hand-raising and queuing of questions. The seminar or event usage model also benefits from having a meeting archive and replay system for participants who can't attend a meeting or want to review an archive. These same platforms subsequently are adapted to serve smaller groups engaged in more collaborative activities.
Net Results of Web Conferencing Vendors:
Click on image to download and enlarge.
Other platforms were designed from the ground up to bring people together in a peer-to-peer setting to collaborate toward a shared goal. In addition to the central actors in the collaboration, passive participants are allowed to observe. This scenario was adopted for our usability tests. Team members reviewed and made edits to files together, and saved the changes.
Despite only a few minor weaknesses, the services tested were highly functional in terms of supporting on-screen collaboration. Holding a telephone in one hand while typing a short text message or making a change on a document is impractical in the minds of all testers. If planning to use the PSTN for voice while doing collaborative meetings, IT should provide appropriate headsets. We liked that most services support emoticons, which appear next to names to indicate agreement or other emotion, but do not interrupt discussion.
One drawback of the event-centric services is that some crowd-management features are still in evidence, although not very useful in a more collaborative scenario. For example, when collaborating, peers expect to be able to interact as they do on a telephone conference call. When using VoIP with Centra eMeeting, the user must push a button to talk. Convoq's moderator-controlled multi-party meeting feature also was challenging to learn and needs enhancements before it can compete with other multi-party VoIP services. We liked that nine of the 11 services let the host see (by way of icons or colors next to participant names) who is speaking, and moderate (give and retrieve the podium and presenter rights). Face-to-Face Meeting and Convoq didn't have the speaker identification feature.
White boards during collaboration allow for on-screen annotation, usually individual remarks added to a white surface or over a graphic image. All services tested (except VidiTel) support whiteboarding, but integration levels differed. In eight of the 11 services, the white board is tightly integrated, but the Genesys white board appears to be a completely separate application that is launched when a white board is requested from a menu. Centra lets a person adjust the refresh rate of the screen manually during a meeting. When the bandwidth is limited, this is a beneficial feature because a slower refresh rate consumes less total bandwidth; for broadband users the default setting is too slow, interfering with collaboration because the typing on the screen and changes in files appear well after the host has referenced the changes. We liked having the ability to see, either through name or initials, the person who makes the annotations (Interwise did this very well).
We were pleased that a few services went beyond bitmapped graphics. Although it required an upload and conversion process, Raindance uses scalable vector graphics technology to show all the images and text in the shared space, ensuring high resolution and independent window sizing for all participants. Macromedia Breeze and Convoq use Flash paper and SWF files (Macromedia's Flash file format) to show any type of media in the shared meeting space.
All the services we tested with video-over-IP support also have the option of suppressing the video in one meeting, across a group of users or on the basis of a corporate policy. But collaborators in a Web conference might want to use the expressions they see on other faces to interpret the support (or lack thereof) of an idea.
With Centra, Convoq, Elluminate, Genesys and Raindance, participants can see only one video window at a time, usually the presenter or host. The host can pass the control to another participant who will then be seen, or the host can choose one person seen by all the others without providing that person full control of the meeting. When there are more than two participants and video is supported, the remaining services support four or more windows, and more closely resemble a multi-point videoconference. We didn't use the video quality as one of the metrics for our final score because of variations in our network bandwidths during testing. Interwise has an unusual implementation of personal video, in which the user must capture a snapshot of himself before sharing/sending out the video to the server. When the person is not speaking, the server reverts to the still image. This conserves bandwidth but reduces the effectiveness of video to detect non-verbal reactions.
We experienced the highest quality video and audio with the Face-to-Face Meeting service. The motion was fluid and the windows resizable. The application monitors bandwidth used against that which is available and performs dynamic rate adaptation. We also really liked the way the Face-to-Face Meeting user can put a call on hold to take another incoming call. The application still needs some work as far as the window management and user interface are concerned; currently an unlimited number of windows can be open at the same time and when one exceeds two or three it is very confusing on the screen.
We felt that, although it didn't have synchronized audio with it and only one participant's video can be seen at a time, the video in Raindance Meeting Edition also offered very high quality.
We like the way Macromedia Breeze offers the user, the host or both independently the ability to choose layouts from pre-configured templates, some of which prioritize the video windows (and others which minimize the video windows and emphasize the presentation or data viewing parts of the screen).
Managing everything
Users can change individual account settings (name, phone number, meeting templates and the like) through portals or preference settings in a runtime application. In general, the services tested let users generate reports, including past conferences, participants in conferences, results of polls, statistics on archive accesses and related historical data. Looking ahead, account holders can use their service portals to view upcoming meetings and create new meetings.
Integration with enterprise communications or workflow systems can greatly reduce overhead associated with populating online directories of collaborators. The large, event-ready platforms such as WebEx, Centra, Elluminate and Interwise let you import a list of people who could be invited to an event or conference. VidiTel and Face-to-Face Meeting use e-mail name convention to identify participants, and Convoq relies on a mix of IM identities and e-mail addresses, depending on how a person will be invited.
Centra, WebEx, Elluminate, Genesys, Macromedia and Interwise also have administrator accounts that let you set policies on behalf of groups of users. Administrator account holders can modify global settings involving meeting archives, network bandwidth utilization, and telephone bridging services for participants who cannot join by VoIP. These same services also offer special fields for billing codes, and let you export meeting reports in formats suitable for accounting.
Personal rich media conferencing through hosted services have matured significantly in the past two years. Mainstream users haven't necessarily discovered the full benefits and aren't clamoring for integrated voice and video yet, but we believe that given the option, they will quickly want to use services such as those we tested. We felt that having the option of bringing users together into an integrated conference regardless of their audio access technologies (a headset or the built-in computer audio systems, or a regular telephone or cell phone) is absolutely the mark all IT managers should be looking to provide their users
About Network World http://www.networkworld.com/aboutus/default.html
Since 1986, Network World has maintained focus on the business of bringing sellers together with highly qualified buyers and strategic decision-makers. Through our newsweekly print publication, online news site, events and executive forums, we help sellers reach the full spectrum of Network IT buyers and decision-makers who are the Network World community.
As the provider of Network IT news and information, Network World has understood the importance of reliable, timely, in-depth information to its community. With every issue, special feature, supplement, online research center, forum, executive conference, technology event and expo, Network World provides Network IT and Business Executives with information where and when they need it.
If you're looking to connect with the Network World community, consult our 2005 Media Guide, or contact a Network World sales representative to learn more about our comprehensive portfolio of premium print, online, event and custom integrated marketing solutions - all designed to help marketing and agency professionals grow their businesses.
All contents copyright 1995-2005 Network World, Inc. http://www.networkworld.com
