Moderator (MDR): MDR-lisa

Guest Speaker (SPK): SPK-name

MDR-lisa: Hello, I'm Lisa Stapleton, your moderator, and I'm here with JavaSoft's Kevin Smith, who is an expert on testing Java applications and for robustness and portability. Some of you might have seen his talk on the same subject at JavaOne. Feel free to start sending questions.

SPK-kasmith: I'm Kevin Smith, one of the test developers at JavaSoft. I thought I would talk about some of the new features of Unit testing that are available in Java that are not available in other Object-Oriented programming languages. I'm also going to talk about some of the tips in System testing for Applets and Applications written in Java.

SPK-kasmith: Let me say a few words about unit testing.

virtual: Yah, I remember last time, you talked about conformance testing, now it's robustness and portability. What're the major issues in these two areas for Java application testing?

abhaloo: Kevin, How would you go about testing a Java application in a distributed environment (we are using CORBA)?

SPK-kasmith: Distributed testing in Java is similar to distributed testing in other environments.

SPK-kasmith: In testing a distributed system, you need to worry about synchronizing all of the pieces of the system so that you have a "starting place" when you begin the test.

Then you need to be concerned with verifying results from the test.

Finally, you need to test for unique distributed system problems such as race conditions, starvation conditions, and deadlock conditions.

With regards to robustness and portability, in Java portability is part of robustness. However, portability is fairly problem domain independent. That's one of the reasons JavaSoft has been working on the 100% Pure Java program. You can think of 100% Pure Java as the measurable/testable aspects of portability.

austinlo: Where is the definition of 100% Pure Java?

SPK-kasmith: There is a programming cookbook being developed that you can use as examples of how to write portable code. There are a number of companies that are writing static class analysis tools that you can use to test for portability problems. The definition of 100% Pure Java has not been finalized, but it includes two parts: static analysis of the classes in your application, and dynamic testing of your application.

I know of two companies that are working on static class analysis tools: SGI and Sun Microsystems, Inc.'s SunTest group (www.suntest.com). I think that the public working paper is available on java.sun.com, although I do not know where on the web site it is.

ktorok: Most of the testing problems I've come across are with the AWT and the GUI. Can any of these tools highlight problems in these areas?

SPK-kasmith: There are a number of aspects to AWT/GUI testing. Some of these aspects are easy to test using the existing GUI test automation tools such as QA Partner and WinRunner/X-Runner.

abhaloo: I am in the process of utilizing SunTest's product called JavaScope to help me out in terms of code coverage. In terms of JavaScope being used in testing in a distributed environment, how can I gather this "coverage info" in one repository/place or does the coverage info get accumulated in different places and require one to manually merge the results?

SPK-kasmith: With regards to using JavaScope in a distributed environment, I have never used that code coverage tool in a distributed environment, so I really cannot say how to aggregate coverage metrics. That's a question better directed to the SunTest folks.

virtual: How do you use those tools? Give us an example of typical usage and the sample benefit.

SPK-kasmith: Getting backt to the AWT, the bigest issue is that it has a native look and feel on each platform it is ported to. So for cross platform testing, it is best to use a cross platform test tool like QA Partner or JavaStar.

abhaloo: What about JavaStar (capture/playback tool) offered by SunTest in terms of GUI Testing? How does JavaStar differ from Segue's QA Partner and Mercury's Winrunner/Xrunner?

SPK-kasmith: The big differences between QA Partner, WinRunner, and JavaStar is how the intercept messages to the GUI interface elements. Both QA Partner and WinRunner intercept messages at the OS GUI library level, for example setMessageHook() in MS-Windows. JavaStar interposes between the platform independent AWT layer and the underlying platform native AWT Peer layer. This makes JavaStar a Java specific test tool, but it also makes it more applicable to testing on almost any Java enabled platform.

I've talked about three different types of tools. With static class analysis tools, I usually check for things like calls to non-java library packages like sun.tools.debug. I also check the class string table for strings with lots of slash or back-slash characters in them. With respect to GUI automation tools, I use them just for testing the GUI of an application. I insist that any application that I test has a batch/command line interface so that it is easier to automate tests for the underlying functionality of the application without having to drive it through the GUI.

mmerani: Our application seems to grow the swapper file at a consistant rate in an OS/2 applet implementation. Is there a testing tool to isolate the problem. Also, is there a leak in garbage collection in the JDK 1.0.2?

SPK-kasmith: I don't know of a specific testing tool to isolate the problem with the swapper file. The first thing you want to isolate is whether the problem is being recognized by the Java system, or whether it is buried down in the native implementation. So you want to test the Java system reported free memory and used memory. If these values accurately reflect what is happening with the swap file, then you need to take a look at the native-code part of the implementation on OS/2.

I'm not sure whether the OS/2 has a verboseGC option, but if it does, use that interpreter command flag to see if GC is behaving as you would expect.

ktorok: What about VM testing issues? Even when I stay with the same platform, I see marked differences between VM's in how the GUI is displayed. Is there any way to bring these differences out with the testing tools? I'm not familiar with the tools you mentioned, but will check them out.

SPK-kasmith: GUI display is not really part of the VM, but relies on the native code implemenation of the AWT libraries. JavaSoft is trying to address this portability problem on two fronts: adding lots more AWT conformance tests to the test suite that licensees must pass, and moving more of the AWT implementation into platform-independent code. For now, you should try to test your application on multiple platforms/implementations. And just for your own sanity, try to automate as many of these tests as you can.

Stencill: Is addressing portability through JavaBeans -> ActiveX too off topic?

SPK-kasmith: For robustness testing, the top issues are usability, mean time between failures, and how easy it is for a user to find a problem in your application. The same old testing issues from any other applications development environment.

In Java, with the features of the main() method, it is easier to write Unit tests. With built in support for observer/observed pattern, it is easier to write applications that can be system-tested without driving the GUI, by driving the app through a command line interface.

sellhorn: We have a very multithreaded environment in our Java app. Is it possible to test for memory usage per thread. How about objects? Can I find out how much memory my objects consume?

SPK-kasmith: In terms of portability, ActiveX is designed to be MS-Windows specific, and JavaBeans is designed to run on any Java 1.1-compliant platform. As Java licensees move to Java 1.1, we should see fewer platform differences, and more portability for things like Java beans, and regular applications written in Java JavaBridge is new technology, I'm not sure where it is in the development cycle. The Java Beans folks should be able to help you. Check out their corner of the java.sun.com website.

abhaloo: What about the system testing "tips" that you were going to talk about?

SPK-kasmith: In terms of information per thread or object, each implementation of Java usually has some access to that type of information, but it is platform dependent. I suggest you pick a platform to optimize on and investigate the platform/vendor interfaces that let you peek at this information. Remember, when you start fiddling with this stuff, it is not portable. In terms of the tips, it looks like we are running out of time to go over them in any detail, but if you want more information please take a look at the JavaOne slides for Tech Track talk 30, Testing Java Applets and Applications. http://java.sun.com/JavaOne/sessions/slides/TT30/index.html

You can also visit my page : http://java.sun.com/people/kasmith/index.html

MDR-lisa: Thanks so much for being here, Kevin. Kevin needs to leave now, but if people want to hang out, I can leave the forum open for 10-15 minutes or so, so y'all can talk. A transcript will be going up later this week on the JDC.

SPK-kasmith: Thank you, Lisa, for inviting me.

Thanks to everyone else for attending.