Customer Profile

e-Travel, Inc., a wholly owned subsidiary of Oracle Corporation (NASDAQ: ORCL), is the leading Internet-based corporate travel management solution. e-Travel is headquartered in Waltham, Massachusetts, and offers a comprehensive application called e-Travel from Oracle in the business-to-business (B2B) e-commerce segment. e-Travel allows corporations to plan and book travel – airplane, rental car, hotel, and rail reservations - using a standard Web browser. e-Travel includes modules for travel planning and reservation purchasing. Two other travel-related products available from e-Travel are: a module called Self Service Expenses, which handles travel reimbursement; and e-Travel Intelligence, which offers travel analytics and reporting.

The e-Travel solution enables companies to reduce and control travel costs, increase service levels for corporate travelers, and strengthen relationships with travel suppliers. e-Travel’s service can both dramatically reduce the administrative costs of corporate travel and increase compliance with travel policies. e-Travel achieves this by offering travel planning coupled with comprehensive business intelligence of corporate travel and entertainment expenses – typically the third or fourth largest cost for most corporations. e-Travel offers the ability to reduce the cost of a travel itinerary from an industry average of $50-$75, to $25 or less. With Self Service Expenses, you can automate employee travel expense reimbursement and reduce the cost of processing expense reports from an industry average of over $35 per expense report, to $10 or less.

e-Travel has competition, but primarily in the business-to-consumer (B2C) market segment – a segment e-Travel does not target. e-Travel is the only travel management application for the corporate market that is independent of travel vendors (like American Airlines, or SABRE) or travel agencies.

Quality Assurance at e-Travel, Inc.

QA responsibility at e-Travel falls under two departments - development and QA; both departments comprise the Application Development Group. Roger Kirkhart, Principle Quality Assurance Engineer, works in development in the QA Automation Group. Roger has been at e-Travel for two years and in his current QA Automation position for one year. Roger’s career has spanned 27 years, with the bulk of his experience falling within the travel, telecommunications, and high-tech industries. Since 1990, he has concentrated on travel automation systems. He worked for four years at WORLDSPAN, prior to his employment at e-Travel, where he was Corporate Sales Manager for WORLDSPAN’s National Accounts group. The National Accounts Group marketed the services of WORLDSPAN’s Computer Reservation System (CRS) to corporations. While at WORLDSPAN, Roger led the first initiative into automated travel booking applications. His technical background includes: Perl, C++, Java, Windows, DOS, Novell, several RDBMS’, C, BASIC, and many other hardware platforms and technologies. Roger has a BS degree in Business Management from the University of Cincinnati. Finally, he has seven children; two of who also work at e-Travel, including one, Jayna, who works in QA Operations.

Besides having extensive experience in the travel industry, Roger also has tremendous enthusiasm. In fact, during the interview, he pulled a small card out of his wallet with a system schematic on it, "I was involved in spec’ing out an automated travel application like e-Travel’s back in the early 90’s, including the automated expense function. I have dreamed of this stuff for a long time. Now, in working on e-Travel’s application, I am seeing this dream come to fruition."

Roger’s QA responsibilities revolve around the procurement of QA tools and technologies, as well as the development of automated tests, for the e-Travel application. He uses SilkTest and SilkPerformer to develop QA tests. He then installs the tests onto a test machine so that he or the QA operations group can perform smoke tests on each new build. Once the smoke test passes, QA Operations commences with a more extensive testing process developed by Roger’s QA Automation group – also utilizing the Silk products.

Roger explains how e-Travel, a Web start-up, became an Oracle subsidiary. "We were founded in 1995 by three entrepreneurs who operated out of an old train depot in Concord, Massachusetts. The depot is an old historic landmark. A funny aside, e-Travel employees conducting business on the phone had to cover the receiver when trains passed by. Oracle became a customer and when they were able to show savings of $50 million dollars by using e-Travel, they decided to buy us. We were purchased by Oracle in March of 1999 and spun off as a wholly owned subsidiary in November of 1999."

The e-Travel application is written in C++, Java, and JavaScript and supports Windows NT, Windows95, and Macintosh platforms. Supported browsers are Netscape and Internet Explorer 3.0.2, and above.

Roger reports to Brian Mottershead, Vice President of Engineering; QA Operations is headed by Linda Collins, Manager of QA, and includes three additional QA professionals: Noorain Buxamua, Jayna Kirkhart, and Rene Morrow . Roger explains the qualifications necessary to work in QA at e-Travel, "Experience in the travel software industry is extremely helpful, as the travel business is such a complex one. And, definitely experience in automated QA testing. The most important attribute to have is to understand QA methods and procedures. As all of Segue’s customers know, QA mandates a different mindset than traditional development or some other systems-related area."

The Release Cycle at e-Travel and Silk Selection Process
Roger explains the release cycle, "Development creates weekly builds. These are typically minor features and service releases. Major milestones occur monthly; these releases include key features. We were originally live on the Web in 1997 and our current version is v. 3.7. We provide QA support here, in Waltham, for the e-Travel application only. QA services for Self Service Expenses are provided out of our offices in Redwood Shores, California."

Prior to licensing SilkTest in December of 1998 and SilkPerformer in May of 1999, all testing was done manually. And, as Linda Collins, Manager of QA, says, there was "lots of pain" involved in so doing. Linda explains the dramatic productivity increase since the implementation of SilkTest, "SilkTest allows us to delve much deeper into our software testing. For example, one reservation transaction for a hotel chain involves testing a reservation for 400 different hotels. Manually, that used to take us days. And, that’s only one hotel chain – of the thousands with whom we deal. You can imagine the testing nightmare that was for us. Now, using SilkTest, that one test can run in an hour."

SilkTest was an easy decision for e-Travel. Roger elaborates, "Our engineering manager was familiar with the product from previous experience. Several of our engineers had used it in prior positions and knew it to be the technically superior solution for QA testing. Additionally, back in 1997 it was the only solution for Web testing. At the time, no other vendor was even talking about Web QA testing. We implemented SilkTest and have now been live with it through three major releases and countless monthly releases. We have been slower to implement SilkPerformer, solely because of resource constraints. Although we have just implemented SilkPerformer, we have already made tremendous improvements in our application performance. The information SilkPerformer returns to us in our performance testing results is extensive and has allowed us to pinpoint potential bottlenecks in e-Travel’s application."

Implementation Gains Realized with Star Quality
"Because of our internal resource constraints, we have used a local Segue consulting partner, Star Quality, throughout our SilkTest implementation. I met Kelly Adams [ed. note: Kelly and Brian LeSuer are the founding partners of Star Quality.] at a local Segue User Group (SUG) meeting. Shortly after the SUG meeting, Kelly came to e-Travel and presented Star Quality’s methodology to us As it turned out, she and Brian LeSuer had been doing some work with another Oracle division right across the street from us. In checking with our sister division, I got an excellent referral on their work. I liked the methodology and approach, so we proceeded. Both Kelly and Brian were here one day a week, over a ten-week period. They automated the testing of a section of our application called Web Admin; it is our configuration system for travel policies and encompasses Java, database, and Web technologies – it’s a pretty difficult module to automate in a short period of time. We hadn’t had the chance to touch it, so even though we had comprehensive tests on the travel reservation side, we couldn’t be sure of the integrity of the policy/configuration system that was driving the reservation transaction decisions. When Star Quality was finished, we had a smoke test on the entire module. Star Quality added a lot of value and I can’t speak highly enough of both Kelly and Brian’s knowledge. They did a superb job for us."

Issues in the Testing Process
The thorniest testing issues faced at e-Travel revolve around the complex combinations of systems to which e-Travel must interface. Roger explains, "All of the multitude of vendors – airlines, car rental agencies, railroads, and hotels – to which we must interface present their information in a different way. There are few industry standards for reservation information. Add to this the real-time nature of our transactions and there is a level of testing complexity that most organizations don’t have to deal with. We are booking actual seats on planes – and availability can change by the second. And, not only are we handling the initial reservation, but we also have to deal with travel cancellations and changes."

"Segue’s tools have helped us in testing because we can quickly test the multitude of vendor targets and systems configurations with which we interface. We have realized such huge jumps in productivity and test coverage because of our SilkTest automation. We can get deeper, more quickly, in our testing and really test functionality, data integrity, and performance - not just simply data entry."

"One example of a gain that quickly comes to mind with SilkPerformer was a handle count issue we had. Handles, as you probably know, are used to keep track of files, memory locations, or data items. Our handle count was increasing and we didn’t know it. We had a handle leak which SilkPerformer allowed us to identify. To isolate and identify this problem, we used Silk Performer to produce ten times the load on our system that our largest customer generates. One of our great development engineers fixed the problem quickly and we celebrated the fantastic teamwork we had between development and QA in working on the problem."

Kelly Adams, Star Quality founder, further discusses the handle issue, "The e-Travel handle leak is very typical of the problems our clients find when they deploy SilkPerformer. Often, these problems are isolated when running as few as five or ten simulated users. Although the maximum number of users purchased is critical for verifying that the system can handle the anticipated load, even small simulations find bugs which are not otherwise found - despite exhaustive manual testing."

Roger continues with other improvements realized throughout the Silk implementation, "On the SilkTest side, we have made tremendous gains in productivity. Our first major test cycle with SilkTest on e-Travel, v. 3.5, took us six months to complete, even though we had modules working within weeks of our implementation; much of this was definitely due to learning curve issues. Our second cycle, for v. 3.7, was three months; this time, we were over the learning curve and we are able to re-use code we had already developed. Our third major release cycle for v. 4.0 is going on right now; I expect this cycle to take about six weeks and cover even more ground in testing depth than our prior cycle. Cycle time for each release’s automated test creation has been cut in half with SilkTest and yet we are doing more testing than ever before. From here on out, we can just continue to grow test coverage because of the Star Quality methodology we implemented, along with Segue’s tools. Silk is so extensible – you don’t lose the efforts you put forth initially in building tests. You can build on top of your previous efforts and just continue to delve more deeply into the product with each test cycle. Also, because of Silk’s portability, we have developed one set of scripts to test on most of the platforms, technologies, and browsers that we support. That, in itself, is a huge gain for us in the QA process."

Kelly Adams further describes the SilkTest benefits e-Travel has realized, "Like most e-commerce Web applications, e-Travel’s applications are very data driven and lend themselves to a high level of test automation. Using SilkTest, e-Travel has not only been able to rapidly develop a broad test suite, but they can now easily expand it just by adding more data combinations to their test plans. e-Travel has also begun using SilkTest to tackle complex verification tasks such as parsing rolling server logs and interpreting graphical airline seating charts to determine seating availability for reservation transactions. These tests calculate the expected results rather than having the results stored as part of the tests. This structure greatly streamlines the process of adding new tests."

Roger summarizes the discussion of QA quality improvements, "With SilkPerformer and SilkTest, we have the luxury of being able to improve both performance and reliability at the same time. We have learned so much about our application and the Web, thanks to Silk. It’s been fantastic."

Remaining Issues
Roger explains the challenges yet to come, "With all that we have done, we are still only part of the way in our quest to attain the ‘ultimate’ testing environment. Silk has enabled us to test broadly – we can now quickly test across our entire application. Our next priority is to test more deeply into our application. We need to look more closely at things like: databases, activity logs, user components, and application performance. This will be easy with Silk – and, again, because of the gains we have made with Segue in our QA automation efforts, we now have the luxury of tackling these areas."

"We have a new challenge coming up in our v. 4.0 release. It will utilize XML; that is a new technology for us in QA Automation. And, our ultimate challenge, of course, is simply time – we all feel the pressure of having to work on Internet time.

In Summary
Roger looks back over the last year’s automation efforts, "I am proud of all that we have accomplished here. I am particularly proud of the fact that this is the first time e-Travel has had QA automation and that I got to implement it. It’s been a lot of fun and a lot of work, but I have personally learned a lot. Best of all, we have been able to improve the quality of the application and find problems we otherwise would not have known about. I also love ‘wowing the engineers’ – and we have done that with our automated tests. It’s not easy to impress development engineers, but we have. We have built a very respected QA group internally and that feels great!"

In closing, Roger had some thoughts he wanted to share with our readers, "First, I cannot emphasize enough the importance of management support for the development of an automated QA process. We’ve had great support for our effort and a management team that understands the benefits to and business payback from QA automation; there is a very real ROI. Secondly, my advice to test software developers, please make it simple and show results for your efforts quickly. And, finally, QA users need to tap the myriad of resources available in the QA community: go to user group meetings, visit QA Web sites, read the newsletter [ed. note: we’re all for that!]. There are so many resources out there that can help further your knowledge – make the effort to learn from them."