Release Notes with Groovy and JIRA REST API: Part II – Separate responsibilities

A while ago I described how to get release notes from JIRA with Groovy, so I thought it’s time to do a small follow up which is long overdue :-).

Let’s split things up to manageable classes with their own responsibility:

  • a JIRA REST client which encapsulates connecting to my JIRA instance, such as (HTTP) getting or posting from and to an url while using the proper credentialsgroovy-logo-medium
  • a JIRA query builder, responsible for building JQL snippets to be used for the JIRA search REST API

We’ll start at the bottom, so we’ll build upon the classes we already have.

Just Being Typesafe

Although a lot of the date we’re getting back from JIRA can be used as-is in Groovy, I decided to make the status of a JIRA issue, e.g. “Open”, into a trivial enum.

enum JiraStatus {
    OPEN("Open"), IN_PROGRESS("In Progress"), RESOLVED("Resolved"), REOPENED("Reopened"), CLOSED("Closed");

    JiraStatus(String value) { this.value = value }
    private final String value
    String value() { return value }
}

I’ll stick to Strings in the remainder of this post 🙂

Building a JQL Query

Queries you can execute within JIRA itself in the Issue Navigator can also be performed through the REST API. Because there are several elements which constitute a JQL-query I decided to use the builder pattern to programmatically create queries through the JiraQueryBuilder.

import groovyx.net.http.RESTClient
import JiraStatus

/**
 * Builder for a JQL-snippet, used for the JIRA search REST API.
 */
class JiraQueryBuilder  {

    private String project
    private int[] sprintIds
    private String sprintFunction
    private String[] issueKeys
    private String[] issueTypes
    private String[] excludedIssueTypes
    private String[] resolution
    private JiraStatus[] status
    private String[] labels
    private String fixVersion
    private String orderBy

    /**
     * Create a new query builder, querying by default everything for specified project, ordered by key.
     */
    public JiraQueryBuilder(String project) {
        assert project
        this.project = project
        withIssueTypes("standardIssueTypes()")
        withOrderBy("key")
    }

    JiraQueryBuilder withSprintIds(int...sprintIds) {
        this.sprintIds = sprintIds
        return this
    }

    JiraQueryBuilder withIssueKeys(String...issueKeys) {
        this.issueKeys = issueKeys
        return this
    }

    JiraQueryBuilder withIssueTypes(String... issueTypes) {
        this.issueTypes = issueTypes
        return this
    }

    JiraQueryBuilder withExcludedIssueTypes(String... excludedIssueTypes) {
        this.excludedIssueTypes = excludedIssueTypes
        return this
    }

    JiraQueryBuilder withResolution(String... resolution) {
        this.resolution = resolution
        return this
    }

    JiraQueryBuilder withStatus(JiraStatus... status) {
        this.status = status
        return this
    }

    JiraQueryBuilder withLabels(String... labels) {
        this.labels = labels
        return this
    }

    JiraQueryBuilder withFixVersion(String fixVersion) {
        this.fixVersion = fixVersion
        return this
    }

    JiraQueryBuilder withOrderBy(String orderBy) {
        this.orderBy = orderBy
        return this
    }

    String build() {
        String jql = "project = ${project}"

        if (sprintIds) {
            jql += " AND sprint in (${sprintIds.join(',')})"
        }

        if (sprintFunction) {
            jql += " AND sprint in ${sprintFunction}"
        }

        if (issueKeys) {
            jql += " AND issuekey in (${issueKeys.join(',')})"
        }

        if (issueTypes) {
            jql += " AND issuetype in (${issueTypes.join(',')})"
        }

        if (excludedIssueTypes) {
            jql += " AND issuetype not in (${excludedIssueTypes.join(',')})"
        }

        if (status) {
            // turn (IN_PROGRESS, RESOLVED) into (Resolved, In Progress)
            jql += " AND status in (${status*.value().join(',')})"
        }

        if (resolution) {
            jql += " AND resolution in (${resolution.join(',')})"
        }

        if (labels) {
            jql += " AND labels in (${labels.join(',')})"
        }

        if (fixVersion) {
            jql += " AND fixVersion = \"${fixVersion}\""
        }

        if (orderBy) {
            jql += " order by ${orderBy}"
        }

        return jql
    }
}

Connecting to JIRA

Finally I needed to encapsulate the connection to a JIRA instance, allowing me to (HTTP) GET and POST from and to it, including using the proper credentials.

// require(groupId:'org.codehaus.groovy.modules.http-builder', artifactId:'http-builder', version:'0.5.2')
import net.sf.json.groovy.*
import groovyx.net.http.RESTClient
import static JiraStatus.*

/**
 * JIRA REST client wrapper around groovyx.net.http.RESTClient
 */
class JiraRESTClient extends groovyx.net.http.RESTClient {

    private static final String DEFAULT_PROJECT = "XXX"
    private static final String DEFAULT_SEARCH_URL = "http://jira/rest/api/latest/"

    String username
    String password
    def credentials = [:]

    /**
     * Create a REST client using Maven properties jira.username and jira.password for JIRA username and password.
     */
    static JiraRESTClient create(def project) {

        String jiraUsername = project.properties['jira.username']
        String jiraPassword = project.properties['jira.password']

        if (!jiraUsername?.trim()) {
            throw new IllegalArgumentException("Empty property: jira.username")
        }

        if (!jiraPassword?.trim()) {
             throw new IllegalArgumentException("Empty property: jira.password")
        }

        return create(jiraUsername, jiraPassword)
    }

    /**
     * Create a REST client using provided JIRA username and password.
     */
    static JiraRESTClient create(String username, String password) {
        return new JiraRESTClient(DEFAULT_SEARCH_URL, username, password)
    }

    private JiraRESTClient(String url, String username, String password) {
        super(url)
        assert username
        assert password

        log.debug "Created for user=${username}, url=" + url
        this.username = username;
        this.password = password;

        credentials['os_username'] = this.username
        credentials['os_password'] = this.password
    }

    def get(String path, def query) {

        try {
            def response = get(path: path, contentType: "application/json", query: query)
            assert response.status == 200
            assert (response.data instanceof net.sf.json.JSON)
            return response
        } catch (groovyx.net.http.HttpResponseException e) {
            if (e.response.status == 400) {
                // HTTP 400: Bad Request, JIRA returned error
                throw new IllegalArgumentException("JIRA query failed, response data=${e.response.data}", e)
            } else {
                throw new IOException("JIRA connection failed, got HTTP status ${e.response.status}, response data=${e.response.data}", e)
            }

        }
    }

    def post(String path, def body, def query) {

        try {
            def response = post(path: path, contentType: "application/json", body: body, query: query)
            return response
        } catch (groovyx.net.http.HttpResponseException e) {
            if (e.response.status == 400) {
                // HTTP 400: Bad Request, JIRA returned error
                throw new IllegalArgumentException("JIRA query failed, got HTTP status 400, response data=${e.response.data}", e)
            } else {
                throw new IOException("JIRA connection failed, got HTTP status ${e.response.status}, response data=${e.response.data}", e)
            }

        }
    }

    /**
     * Search JIRA with provided JQL e.g. "project = XXX AND ..."
     * @returns response
     * @throws IllegalArgumentException in case of bad JQL query
     * @throws IOException in case of JIRA connection failure
     * @see JiraQueryBuilder to create these JQL queries
     */
    def search(String jql) {
        assert jql

        def query = [:]
        query << credentials
        query['jql'] = jql

        query['startAt'] = 0
        query['maxResults'] = 1000

        log.debug "Searching with JQL: " + jql
        return get("search", query)
    }

Credentials are appended in plain sight to the url, but for internal use I found this acceptable. The search() gets a JQL string passed, appends the credentials and delegates to our own get() which actually performs groovyx.net.http.RESTClient‘s get() to make it happen.

Querying Release Notes

The query to get all Resolved or Closed issues for a certain fix version  can be encapsulated in a getReleaseNotes method, such as seen below.

def getReleaseNotes(String fixVersion) {
  String jql = new JiraQueryBuilder(DEFAULT_PROJECT)
                    .withStatus(RESOLVED, CLOSED)
                    .withFixVersion(fixVersion)
                    .build()
  return search(jql)
}

Done

I’m still kickstarting this script with e.g. the gmaven-plugin which allows me to get my JIRA credentials from e.g. settings.xml, while the fix version for the JIRA query can be supplied as a property on the command-line.

Release Notes with Groovy and JIRA REST API

JIRA is Atlassian‘s kick-ass issue tracker and in use in many projects I’ve been in. In my current project we use JIRA’s release notes after a certain version has been released, but some departments need the release notes fixed inside the created Maven artefact(s). For this we copy the release notes from JIRA into ReleaseNotes.xml to produce a PDF with the docbkx-maven-plugin – a tedious, manual process.

So, I set out to automate a few things: get the release notes from JIRA and convert to DocBook – all programmatically. Continue reading “Release Notes with Groovy and JIRA REST API”

Using Jenkins Build Version With Maven

Knowing which version of your application has been deployed on a certain environment is essential for today’s software development and maintenance. Have you ever needed to support a customer which reported a bug in a feature for which you already provided a patch in version control? That you asked yourself why the heck that fix isn’t on production and you don’t even know where or how to verify this?

There are many ways of versioning (which by itself is a very broad concept) your software but a simple mechanism to track what’s being deployed could be;

  1. Use a buildtool (e.g. a continuous integration server such as Atlassian Bamboo or Hudson/Jenkins) to create your build. Use it to relate the build with the changes and code in it.
  2. Identify the version and make this visible in the application.

You don’t create builds from your local workspace these days, do you? 🙂

Below a simple outline of achieving this with Jenkins, creating a Mavenized webapplication (WAR), passing along a build number and reading it back in application code e.g. a JSF backing bean.

Using a build number

Define a property in your Maven pom.xml which holds the build number which we can overridde externally later on:

<properties>
	<build.number>SNAPSHOT</build.number>
</properties>

Create a manifest

If your application is packaged as a WAR (<packaging>war</packaging>) you can instruct the Maven War plugin to create a manifest file. This is a META-INF/MANIFEST.MF file which we can pass along some meta information about a JAR or in this case, the WAR.

<build>
	<plugins>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-war-plugin</artifactId>
			<configuration>
				<manifest>
					<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
				</manifest>
				<archive>
					<manifestEntries>
						<Specification-Title>${project.name}</Specification-Title>
						<Specification-Version>${project.version}</Specification-Version>
						<Implementation-Version>${build.number}</Implementation-Version>
					</manifestEntries>
				</archive>
			</configuration>
		</plugin>
	</plugins>
</build>

You could leave out some stuff, but the essential part is Implementation-Version which uses the ${build.number} property. This will create a MANIFEST.MF which sort of looks like the following:

Manifest-Version: 1.0
Implementation-Version: SNAPSHOT
Built-By: jenkins
Build-Jdk: 1.6.0_31
Specification-Title: Example Project Name
Created-By: Apache Maven
Specification-Version: 0.0.1
Archiver-Version: Plexus Archiver

If you don’t overridde build.number it’ll default to SNAPSHOT. Well, for now this doens’t add much value, but now we can have Jenkins set it for us too.

Jenkins Environment Variables

If you would translate the above mvn commandline to a Jenkins Maven job you can use several of Jenkins’ environment variables to pass along in properties. E.g.

  • BUILD_NUMBER – The current build number, such as “153”
  • BUILD_ID – The current build id, such as “2005-08-22_23-59-59” (YYYY-MM-DD_hh-mm-ss)
  • SVN_REVISION – For Subversion-based projects, this variable contains the revision number of the module.

You can take anything from the list (if applicable, such as SVN_REVISION when using Subversion) and pass it along as property. Let’s take BUILD_NUMBER as our build.number property: mvn clean install -Dbuild.number=${BUILD_NUMBER}

In Jenkins it would look like

Jenkins Maven Build With Properties

If you would run the Jenkins job again, its current build bumber would be used and end up in the manifest file.

Manifest-Version: 1.0
Implementation-Version: 153
Built-By: jenkins

Reading the version

To wrap up, you obviously need to make this version visible in a running application. You could add the version to the HTML header in comments or display it on some sort of health page. For working with manifest files you can read the Java Manifest Files Tutorial. To read from it use java.util.jar.Manifest and java.util.jar.Attributes as you can see in the (JSF) example below.

ExternalContext application = FacesContext.getCurrentInstance().getExternalContext();
InputStream inputStream = application.getResourceAsStream("/META-INF/MANIFEST.MF");
Manifest manifest = new Manifest(inputStream);

Attributes attributes = manifest.getMainAttributes();
String version = attributes.getValue("Implementation-Version");

Conclusion

There’s obviously more to it than meets the eye…or your requirements. You can now see which build of the application is running on a environment, and going back to your CI server – which generated the build number – and relate it to the code inside the build. We haven’t used Maven’s own project version, release plugins or other fancy tools yet, but if you start simple you can always expand into more elaborate schemes to suit your needs.

GWT Compiler Is Running Out of Memory

As part of evaluating an open-source framework which uses Google Web Toolkit I ran into some of the familiar issues I already had a few times earlier with the GWT (GWT 2.4 or 2.5) Maven plugin compilation running out of memory. You know, when you see…

…at the start:

[INFO] #
[INFO] # There is insufficient memory for the Java Runtime Environment to continue.
[INFO] # Native memory allocation (malloc) failed to allocate 32744 bytes for ChunkPool::allocate

…or during compilation:

[INFO]    Compiling 12 permutations
[INFO]       Compiling permutation 1...
[INFO]       Process output
[INFO]          Error occurred during initialization of VM
[INFO]          Could not reserve enough space for object heap

This post is basically a compilation of the answers found online, and a quick reminder for myself and others who are running into similar problems with the fantastic GWT compiler :-).

1. First of all, check if you actually need to compile everything! Skip some browsers or locales – saving all kinds of permutations – if you don’t need it for development. Use the tips about how to speed up the GWT compiler and GWT compilation performance. There’s also an existing FAQ about GWT Debugging and Compiling.If you have control of your machine, cut down on other running applications and processes while you’re compiling.

2. Pass along some more memory

…to Maven, e.g.:

MAVEN_OPTS=-Xms1024m -Xmx1024m -XX:MaxPermSize=256m

…and/or to the gwt-maven-plugin, e.g.:

<configuration>
  <extraJvmArgs>-Xms768m -Xmx768m</extraJvmArgs>
</configuration>

You’ll have to play a bit with the values, depending on the machine you’re working on and the actually available working RAM.

Add some permgen space, which is used when GWT compiles Java into Javascript. E.g.

-XX:MaxPermSize=1024m

3. Try lowering the number of workers to a number explicitly below your available cores/cpu’s. If this number isn’t defined explicitly, the gwt-maven-plugin will set these to the number of available cores/cpu’s. Less workers = less parallellisation = less memory overhead.

<configuration>
  <extraJvmArgs>-Xms768m -Xmx768m</extraJvmArgs>
  <localWorkers>3</localWorkers>
</configuration>

4. Try having the “workers” work in threads, instead of spawning new processes.

<configuration>
  <extraJvmArgs>-Xms768m -Xmx768m -Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev.ThreadedPermutationWorkerFactory</extraJvmArgs>
  <localWorkers>3</localWorkers>
</configuration>

Hopefully this will save someone some valuable time finding solutions. If you have other options, feel free to add them in the comments.