Intelligent Enterprise

Simple, right? Initially, we thought so, too. But after testing, we came to realize that nothing relating to BI is simple, at least not yet. We encountered problems involving dirty and incomplete data, the inability of products to correctly deduce time-based fields from a simple date column, and faulty queries arising from incomplete joins that resulted in erroneous data being reported. Still, once configuration was completed and we'd worked around these issues, we discovered all sorts of golden nuggets lying in the streambed of our data. And in most cases, we didn't have to work hard to get at them. We easily found the top three revenue-generating states (Ohio, Texas and Pennsylvania) and the most popular category of widgets (wireless and storage in both 2004 and 2005). We also learned that sales of our Widget 100 model decreased exponentially during the first three quarters of 2005.

Setting Up Camp

With the exception of Applix's and Microsoft's, the products we tested required deployment within an application server; happily, all supported a wide variety of industry-standard choices, including BEA WebLogic, IBM WebSphere, IIS, JBoss and Tomcat. Microsoft SQL Analysis Services is an extension to SQL Server 2000, with client access provided through its Enterprise Manager; Applix's TM1 is a Windows-based server.

Installation of all products was a breeze, though we found configuration painful with Information Builders' WebFocus and MicroStrategy's eponymous product, both of which required us to configure multiple administrative applications for functions such as report distribution. We suggest taking advantage of training options for these vendors' products.

BI Vendors At A Glance Click to Enlarge

Generating reports with every product proved simple, and experience with existing enterprise reporting applications, such as Crystal Reports, will transfer easily to all products we tested. However, more difficult tasks like integration with portals, as well as more complex report applications, will require additional time and effort. Information Builders' WebFocus, you'll need to learn its scripting language.

And as the alarms coming from our lab's APC racks indicated, report generation requires quite a bit of power and CPU utilization. The more data a report retrieves, the more CPUs you'll need. Where possible, administrators will want to defer generating reports known to be unwieldy--only Cognos let us cancel long-running reports and free our CPUs.

Polished Reports

MicroStrategy 8 was perhaps the most exacting of the products we tested regarding the organization of data within our database. Although all were touchy about working with dirty data, all but MicroStrategy worked with our existing data structure to create cubes and reports. In contrast, MicroStrategy is very much data-warehouse-aware and expected specific fields to exist; it also required us to create and use separate lookup tables to join across tables. Even providing these tables did not ensure success, and we were unable, even with assistance, to resolve the problem of joining a single--but important--table during the modeling process. This snafu prevented MicroStrategy from correctly aggregating data for our simple report.

Indeed, report generation was by far the feature that differentiated the products we tested. Some products, notably those from Applix, Information Builders, Microsoft and MicroStrategy, provided almost no ad hoc query capabilities other than those offered through their respective fat-client report design and modeling tools. Microsoft's SQL Server 2000 Reporting Service was the worst of the bunch, offering only ad hoc queries from within its Visual Studio .Net 2003 plug-in as opposed to the preferred drag-and-drop Crystal Reports-style interfaces provided by rivals.

At the other end, Cognos took report creation to new levels with its Web-based Report Studio and Query Studio. If your business analysts can't use Cognos' end-user-focused tools, check their pulses. Actuate 8's report creation, though not as user-focused, was equally impressive and intuitive, but we preferred Cognos' deep integration and Web-focused vision.

Information Builders' WebFocus report designer was Crystal Report-esque but more awkward to use than those from Actuate and Cognos. One of the gems hidden within WebFocus, however, is the ability to autogenerate an "accordion report," which represents hierarchies in an Explorer metaphor and let us browse through reports level by level with ease. Actuate 8 offers similar report styles with its e-Spreadsheet, an Excel-like piece of software minus the limitations on row counts.