Code Coverage Using Cobertura

Share |

What is Cobertura?

Cobertura is a free Java tool that calculates the percentage of code accessed by tests. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage.

Features

• Can be executed from ant or from the command line.
• Instruments Java bytecode after it has been compiled.
• Can generate reports in HTML or XML.
• Shows the percentage of lines and branches covered for each class, each package, and for the overall project.
• Shows the McCabe cyclomatic code complexity of each class, and the average cyclomatic code complexity for each package, and for the overall product.
• Can sort HTML results by class name, percent of lines covered, percent of branches covered, etc. And can sort in ascending or decending order.

Introduction

In a recent interview, James Gosling mused: "I don't think anybody tests enough of anything" (A Conversation with James Gosling).

The fact is, software today is horrendously under-tested. Software maintainence costs are spiralling because few pieces of software can be changed with any degree of confidence that new bugs aren't being introduced.

Without continual change and improvement, software is unable to continue meeting its users' needs. But without comprehensive automated test-suites, such change can also expose users to very undesirable risks: the risk that critical features no longer function as before, the risk of data loss and the risk of a system crash.

Agile software development methodologies are helping improve the quality of software. Test-driven development, where tests are written to test features before those features are added, ensures that every piece of software is coupled with a test-suite.

Why use a coverage tool?

No matter how good such methodologies are, and how diligently they're followed in an organisation, it isn't possible to ensure that software testing is as comprehensive as it could be. That's where tools can help.

Cobertura is a free, simple and easy-to-use tool that will complement your existing Java development practices. It helps you discover exactly where your software is being tested and, more importantly, where it isn't. Cobertura will help you to view your software from a number of levels, from the entire system right down to an individual line of code.

Why use Cobertura?

Sure, there are other coverage tools around, so what makes Cobertura different?

Coverage analysers work by adding instrumentation. For Java, coverage analysers fall into three categories: those that insert instrumentation into the source code, those that add instrumentation to the Java byte-code, and those that run the code in a modified JVM. Cobertura adds instrumentation directly to the bytecode. We feel this is the best approach, since it does not require a modified VM, but still retains a big speed advantage over having to compile all your source code twice.

Secondly, Cobertura is easy to integrate with Apache Ant. It comes with it's own Ant task definitions for you to use. You can choose to instrument any code you wish, from a single class to an entire system.

Finally, Cobertura is completely free and not time-locked, so you can begin to use it today, without having to worry about what to do at the end of an evaluation period. What's more, we believe Cobertura is so easy to use, you'll be up and running in no time. So why not spend the next 15 minutes getting up and running with Cobertura. In that time, you'll certainly find out which parts of your code are completely tested, and where your code could do with a bit more testing.

Getting started

This chapter assumes you are already familiar with the Ant build tool from Apache. If you're unfamiliar with Ant, you can find out about it at http://ant.apache.org/.

Adding the Cobertura custom tasks to Ant

Cobertura seemlessly integrates with Ant using custom tasks. But before you can use these new tasks, you have to declare them in your Ant build file, typically named build.xml. This is done using the taskdef element, as shown below. Place this element anywhere in your Ant build file.

Now we're ready to start using the Cobertura tasks.

Adding instrumentation to your classes

Cobertura works by inserting instrumentation instructions directly into your compiled Java classes. When these instructions are encountered by the Java Virtual Machine, the inserted code increments various counters so that it is possible to tell which instructions have been encountered and which have not.

You instruct Ant to create instrumented versions of your classes using the instrument task. The example below assumes your classes are in the directory build/classes.

Running an instrumented application

Once your classes have been instrumented by Cobertura, you can continue testing your application as you would normally via the JUnit task.

Simply include a classpath entry for the instrumented classes before any reference to the original classes. This will ensure that the instrumented classes are loaded in preference to the original classes.

...

The instrumented classes found in ${build.instrumented.dir} will be loaded before those found in ${build.classes.dir} ensuring that the instrumented classes are used by JUnit.

An instrumented class serializes information into the file cobertura.ser. Any existing information found in this file will be merged with the current information. In this way the instrumentation for several sessions of a running program can be merged together, producing a single coverage report. For example, the instrumentation from unit and functional tests can be merged together to produce a single coverage report.

Cobertura reporting

An instrumented class serializes coverage information to the file cobertura.ser. Using the report tag, Cobertura can generate coverage reports in either HTML or XML format.

HTML coverage report

The default format for a Cobertura report is HTML.

XML coverage report

The type of report generated is controlled by the format attribute of the report tag, which may be either html or xml. If the format is not supplied then it defaults to html.

Cobertura session merging

Sometimes it is necessary to merge several Cobertura sessions together, for example, to produce a single consolidated coverage report from several different test runs of an application.

Session merging with the merge tag

The above fragment from an ANT build file, will merge together any serialised instrumentation that has been generated by Cobertura and produce a single consolidated instrumentation record.

Taking control with Cobertura and check

All too often testing is something that is left until the end of the development cycle. For example, a coverage report is run on a weekly basis over the codebase, only when the coverage metrics fall below a certain pain threshold are the developers directed to increase their test coverage and quality.

With Cobertura, development teams can choose to enforce test driven development on their codebase, by using the cobertura-check tag in their ANT build scripts.

After each instrumented unit test sequence, Cobertura can do a health check to ensure that the codebase is being tested to the standard that has been demanded by the team. If coverage standards fall below the criterea set by the team, cobertura-check will fail the build.

Our intention with cobertura-check is to ensure that test driven development practices are being followed by the entire development team. Inadequate testing will result in an immediate automated build failure, without having to study the entire coverage report.

This article is taken from official website of Cobertura http://cobertura.sourceforge.net
You can visit that for more information about coberura 

Share |

Creating Nightly build, coverage, JUNIT testcases using LUNT BUILD

Share |

Official Website for Luntbuild is http://luntbuild.javaforge.com/

Prerequisites

1) Install Tomcat
2) Install Apache Ant
3) Add jar files for Antelope Apps and Antelope Tasks or any other required jar files of tasks to /lib
4) Start Tomcat
5) Install all dependent softwares for the project

Luntbuild Installation

1) Execute installation jar file

• java -jar luntbuild-x.y.z-installer.jar

2) Specify luntbuild installation directory

• d:\ProgramFiles\luntbuild

3) Provide following parameters for customizing web application

Session time out [min]: 30
Luntbuild Account password: luntbuild (luntbuild is admin account for Luntbuild web application)
Path to war deploy dir: E:\tomcat\apache-tomcat-5.5.20\apache-tomcat-5.5.20\webapps (tomcat webapps dir)

4) Provide database configuration information

5) Provide information for LDAP authorization (if required) (Not required in out case)

Configuring Luntbuild

1) Login to Luntbuild web GUI (http://localhost:8080/luntbuild/) using default username and password (luntbuild, luntbuild (or what ever password you have given while installation))

Note: The web GUI will automatically login as anonymous user, hence first you need to logout

2) Go to Users tab and create necessary users

3) Go to Projects tab

4) Create new project - Click on Icon which is bellow Logout in top right side of the page to add new project

5) Fill the following details for creating new project

Name: Name of the project

Description: Nightly Build

Project admins select the users who should be assigned the role of 'project admin'.

Project builders select the users who should be assigned the role of 'project builders'.

Project viewers select the users who should be assigned the role of 'project viewers'.

Notification methods : Email (You can select other methods also)

Notification users select the users who will get notified for the builds of this project.

Variables nightlyVersionIterator=1
releaseVersionIterator=1

Log level verbose

6) After filling necessary details click on save button to save new project

7) Now got to VCS adaptors tab and click on icon bellow logout in top right side of the page to add new Version Control System for the project

8) Fill the following details for the VCS Adaptors

Version Control System: Dynamic Clearcase

View tag: luntbuild_myproject

Clearcase view stgloc name

Explicit path for view storage For Windows machine share some directory on the machine where you want to create a view storage for luntbuild
and provide the shared directory location with view file name as follows:

\\mypc\sharedDir\viewname.vws

Config spec Provide clear case config spec for the project
Note:
1) Here you can copy & paste the config specification of your existing view
2) For windows machine remove the directory name where the View is mapped and give only relative path in config spec.

Modification detection config

Options for mkview command

Path for cleartool executable \bin

clearcase_home = D:\atria\ClearCase or C:\Program Files\Rational\ClearCase
Note: Only provide the path up to where cleartool executable is stored and do not include the cleartool.exe in the path
and do not specify like this D:\atria\ClearCase\bin\cleartool.exe

History format parameters

MVFS Path : M:

Project path:

Quiet period : 5

9) Click on the save button after filling necessary details to save the VCS configuration

10) Now go to Builders tab to create the builder for the project

11) Click on the icon bellow Logout at top right side of the page to add new Builder to the project

12 ) Fill the following details for the builder

builder type: Ant Builder

Name: nightlybuild_myproject

Command to run Ant: E:\apache-ant-1.7.0\bin\ant.bat

Build script path:

Build targets: main

Build properties:

artifactsDir="${build.artifactsDir}"
junitHtmlReportDir="${build.junitHtmlReportDir}"
coverageHtmlReportDir="${build.publishDir}/coverage_html_report"
properties.files="file name containing properties"
clearcase.viewtag="luntbuild_myproject"
luntbuild.clearcase.vob.prefix=""

Environment variables: JAVA_HOME=C:\Program Files\Java\jdk1.5.0_09
ANT_HOME=E:\apache-ant-1.7.0
ANT_OPTS=-Xmx512m

Build success condition result==0 and logContainsLine("BUILD SUCCESSFUL")

13) After filling necessary details click on the save button to save the Builders details

14) Then go to Schedules tab and click on icon bellow Logout to add new Schedule for the project

15) Fill the following details for the schedule

Name: Schedule_myproject

Description:

Next build version nightlybuild-${#currentDay=system.(year+"-"+month+"-"+dayOfMonth),#lastDay=project.var["day"].setValue(#currentDay),#dayIterator=project.var["dayIterator"].intValue,#currentDay}.${project.var["dayIterator"]}

Work directory

Trigger type cron

Cron expression: 0 0 6 * * ?

Build necessary condition: alwaysIfFailed or vcsModified or dependencyNewer

Associated builders : nightlybuild_myproject

Associated post-builders:

Build type: clean

Post-build strategy: do not post-build

Label strategy: do not label

Notify strategy : notify always

Schedules the current schedule
depends on:

Dependency triggering strategy: do not trigger any dependent schedules

Build cleanup strategy: keep build by days 7

16) After filling all necessary details click on the save button to save the schedule

17) You can also configure Login mapping of luntbuild users with the VCS login
for that go to "Login mapping" tab and click on the icon bellow Logout to create new VCS login mapping entry

For Manually building the project

1) Now go to Home > Select Builds tab > click on the Icon which specify the to trigger the build to trigger the build manualy

2) Fill the following details to trigger the build

Build necessary condition

Build as version

Build type

Post-build strategy

Label strategy

Notify strategy

Dependent triggering strategy

When to build Select one the following
Now start building now
After start building after specified minutes
At start building at specified time in hh:mm format

3) Click on the save to save the building details for the project

View the Build status:

1) Go to home > select Builds tab
here you can view all the projects and their current build status

2) You can click on the link in the "Schedule" column for the project to see the schedule details of the project

3) You can click on the link in the "Last Build" column to view the success or failure details of last build of the project

Here you can find build log, revision log and artifacts for the project

Some troubleshooting tips:

1) While adding new VCS adaptors
a) Use new "view tag" which is not being used
For Example, luntbuild

b) Do not specify any thing for "Clearcase view stgloc name"

c) Use shared location on the local machine for "Explicit View path"
For example,\\mypc\luntbuildShared\luntbuild_view.vws

d) While working on windows machine in config spec specify all the path relatively without giving mapped directory name on Unix machine

For Example,

Instead of giving paths in config specs as follows
core projects
element /mydir/myproject/abc/xyz

give paths as follows (i.e. do not use /cntdb at beginning)
core projects
element /myproject/abc/xyz

2) While configuring Builders

a) Give the proper location as per for Command to run Ant (Exact location where ant.bat is stored)

b) Give complete path for "Build script path"

c) Specify following things correctly in "Environment variables"

JAVA_HOME=C:\Program Files\Java\jdk1.5.0_09
ANT_HOME=E:\apache-ant-1.7.0
ANT_OPTS=-Xmx512m

3) If luntbuild is giving the following error while building the project

Error occurred during initialization of VM

Could not reserve enough space for object heap

Could not create the Java virtual machine.

Ant builder failed: build success condition not met!

For the above error you need to increase heap size of the JVM as follows:

a) Go to Home > Projects > Builders

b) Edit the existing Builder

c) In Environment variables" which is ANT_OPTS=-Xmx512m
you can use ANT_OPTS=-Xmx1024m or higher as per the need

Share |